-- Logs begin at Wed 2022-08-03 10:00:16 UTC, end at Wed 2022-08-03 11:05:16 UTC. -- Aug 03 10:28:38.424776 dv-cinder-850836-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 03 10:28:38.433796 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 03 10:28:38.436218 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 3 10:28:38 2022] *** Aug 03 10:28:38.436218 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 03 10:28:38.436218 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 03 10:28:38.436218 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: nodename: dv-cinder-850836-2-iscsi Aug 03 10:28:38.436218 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: machine: x86_64 Aug 03 10:28:38.436218 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: clock source: unix Aug 03 10:28:38.436218 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: pcre jit disabled Aug 03 10:28:38.436218 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: detected number of CPU cores: 4 Aug 03 10:28:38.436218 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: current working directory: / Aug 03 10:28:38.436218 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: detected binary path: /usr/bin/uwsgi-core Aug 03 10:28:38.437242 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: your processes number limit is 47680 Aug 03 10:28:38.437242 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: your memory page size is 4096 bytes Aug 03 10:28:38.437242 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: detected max file descriptor number: 2048 Aug 03 10:28:38.437242 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: lock engine: pthread robust mutexes Aug 03 10:28:38.437242 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: thunder lock: enabled Aug 03 10:28:38.437242 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 03 10:28:38.437242 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 03 10:28:38.488597 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: Python main interpreter initialized at 0x5569ebad59c0 Aug 03 10:28:38.488597 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: python threads support enabled Aug 03 10:28:38.488597 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: your server socket listen backlog is limited to 100 connections Aug 03 10:28:38.488597 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: your mercy for graceful operations on workers is 90 seconds Aug 03 10:28:38.489798 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: mapped 403077 bytes (393 KB) for 2 cores Aug 03 10:28:38.489798 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: *** Operational MODE: preforking *** Aug 03 10:28:38.490019 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: *** uWSGI is running in multiple interpreter mode *** Aug 03 10:28:38.490019 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: spawned uWSGI master process (pid: 80306) Aug 03 10:28:38.490257 dv-cinder-850836-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 03 10:28:38.491891 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: spawned uWSGI worker 1 (pid: 80318, cores: 1) Aug 03 10:28:38.491891 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: spawned uWSGI worker 2 (pid: 80319, cores: 1) Aug 03 10:28:38.491891 dv-cinder-850836-2-iscsi devstack@keystone.service[80306]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 03 10:28:44.173773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 10:28:44.173773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: from cryptography.utils import int_from_bytes Aug 03 10:28:44.174907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 10:28:44.174907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: from cryptography.utils import int_from_bytes Aug 03 10:28:44.192488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 10:28:44.192488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: from cryptography.utils import int_from_bytes Aug 03 10:28:44.193426 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 10:28:44.193426 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: from cryptography.utils import int_from_bytes Aug 03 10:28:44.228610 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 03 10:28:44.228955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 03 10:28:44.229278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] command line args: [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 03 10:28:44.229773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 03 10:28:44.230061 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ================================================================================ {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 03 10:28:44.230430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] admin_token = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.230691 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] config_dir = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.230887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.231076 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] config_source = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.231261 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] debug = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.231469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.231846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.232026 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.232351 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.232555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.232795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] list_limit = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.232979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] log_config_append = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.233164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.233350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] log_dir = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.233563 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] log_file = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.233820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.234117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.234354 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.234627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.234869 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.234869 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.235412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.235412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.236938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.236938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.236938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.236938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.236938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.236938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.238439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.238439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.238439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] publish_errors = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.238439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.238439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.238439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.240294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.240294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.240294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] standard_threads = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.240294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.240294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.240294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.241342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] use_journal = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.241342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] use_json = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.241342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] use_stderr = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.241342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] use_syslog = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.241342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.241342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.242450 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.242450 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.242450 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.242450 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.243148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.243148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.243148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.243666 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.243666 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.244025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.244025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.244369 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.244518 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.244705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.244898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.245152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.245417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.245709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.245961 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.246214 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.246477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.246752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.247014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.247273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.247517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.247758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.248010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.248255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.248515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.248772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.249010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.249274 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.249522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.249787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.249973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 03 10:28:44.250157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.250465 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 03 10:28:44.250648 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.250830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] command line args: [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 03 10:28:44.250972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.251088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 03 10:28:44.251223 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.251415 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ================================================================================ {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 03 10:28:44.251596 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.251785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] admin_token = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.251972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.251972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.252318 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] config_dir = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.252569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.252741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.252920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.253091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] config_source = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.253277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.253518 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] debug = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.253724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.253909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.254129 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.254315 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.254491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.255330 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.255514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.255674 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.255843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.255843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.256192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.256388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.256572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] list_limit = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.256756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.256942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] log_config_append = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.257120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.257311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.257525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.257705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] log_dir = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.257872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.258042 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] log_file = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.258214 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.258411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.258593 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.258764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.258937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.258937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.259268 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.259557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.259744 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.259918 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.260085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.260254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.260435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.260623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.260797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.260959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.261132 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.261309 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.261309 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.261722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.261910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.262086 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.262259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.262436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.262656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.262828 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.263006 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.263182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.263361 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.263361 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.263707 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.263904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.264087 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.264247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.264247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.264247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.264567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.264727 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.264843 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] publish_errors = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.264963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.265129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.265311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.265311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.265666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.265838 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.266013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.266194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.266505 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.266756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.266945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.267123 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.267302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] standard_threads = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.267476 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.267476 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.267832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.268008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.268194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.268371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.268544 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.268725 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.268904 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] use_journal = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.269081 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.269256 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] use_json = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.269437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.269622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] use_stderr = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.269792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.269975 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] use_syslog = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.270148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.270325 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 10:28:44.270514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.270514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.270860 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.271021 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.271180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.271339 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.271496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.271675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.271675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.272002 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.272168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.272331 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.272551 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.272812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.272812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.272812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.272812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.272812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.272812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.273809 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.273975 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.274140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.274140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] auth.external = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.274474 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.274647 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.274830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.275003 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.275191 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.275385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.275573 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.275755 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] auth.password = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.275755 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] auth.token = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.276094 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.276277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.276454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.276631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.276800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.276975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.276975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.277322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.277569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.277696 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.277811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.277811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.278100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.278273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] credential.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.278273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.278588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.278745 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.278922 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.279120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.279120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.279463 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.279655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.279655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.279983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.280173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.280173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.280512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.280684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.280858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.281034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.281222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.281401 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.281401 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.281944 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.281944 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.282305 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.282499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.282671 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.282907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.282907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.283236 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.283423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.283589 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.283757 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.283757 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.284089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.284267 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.284267 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.284627 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.284807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.284985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.285166 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.285166 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.285880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.286091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.286233 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.286485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.286485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.286829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.287010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.287202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.287382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.287566 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.287745 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.287924 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.288088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.288088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.288433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.288623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.288809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.289000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.289179 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.289372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.289372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.289741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.289926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.290120 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.290301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.290493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.290686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.290841 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.291005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.291005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.291353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.291545 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.291731 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.291957 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.292157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.292348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.292532 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.292707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.292879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.293132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.293132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.293448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.293748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.293931 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.294112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.294112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.294477 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.294659 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.294843 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.295024 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.295205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.295397 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.295586 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.295768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.295768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.296105 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.296300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.296492 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.296676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.296871 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.297053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.297230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.297411 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.297602 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.297763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.297763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.298106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.298289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.298472 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.298663 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.298846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.299028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.299028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.299359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.299487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.299677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.299858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.300037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.300217 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.300217 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.300554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.300748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.300930 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.301123 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.301304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.302001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.302181 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.302379 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.302561 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.302753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.302753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.303152 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.303333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.303524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.303703 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.303882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.304054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.304054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.304386 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.304582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.304763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.304938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.305128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.305308 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.305594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.305727 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.305727 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.305990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.306168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.306355 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.306646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.306826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.307004 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.307183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.307363 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.307363 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.307708 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.307891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.308057 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.308241 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.308423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.308599 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.308599 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.308947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.309124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.309303 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.309510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.309688 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.309898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.310078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.310260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.310260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.310537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.310716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.310911 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.311102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.311284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.311472 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.311472 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.311817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.311970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.312111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.312261 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.312440 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.312622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.312799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.312998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.312998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.313395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.313598 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.313772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.313957 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.313957 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.314318 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] auth.external = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.314500 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.314692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.314875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.315059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.315253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.315253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.315541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.315720 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.315916 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] auth.password = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.316105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.316300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] auth.token = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.316503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.316797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.316919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.317746 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.317935 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.317935 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.318302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.318482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.318664 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.318844 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.319011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.319208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.319389 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.319922 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.319922 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] role.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.320270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.320453 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] role.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.320650 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] credential.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.320831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.321012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.321177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.321177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.321504 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.321709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.321912 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.322105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.322286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.322469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.322652 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.322787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.322904 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.323068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.323068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.323475 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.323657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.323851 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.324033 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.324214 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.324392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.324392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.324743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.324922 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.325098 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.325289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.325504 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.325686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.325891 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.326069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.326251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.326445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.326445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.326791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.326984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.327167 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.327353 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.327537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.327725 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.327901 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.328097 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.328097 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.328440 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.328628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.328824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.329010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.329210 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.329397 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.329686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.329872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.329872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.330212 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.330417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.330594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.330780 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.330957 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.331135 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.331135 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.331481 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.331655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.331839 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.332013 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.332188 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.332365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.332365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.332813 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.332998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.333180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.333426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.333638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.334030 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.334212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.334212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.334550 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.334733 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.334934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.335115 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.335291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.335478 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.335662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.335840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.335840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.336188 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.336371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.336550 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.336725 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.336902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.337095 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.337276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.337451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.337451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.connection = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.338100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.338294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.338476 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.338669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.338853 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.339036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.339036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.339348 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.339511 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.339800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.339985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.340165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.340356 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.340356 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.340685 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.340875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.341052 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.341230 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.341401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.341603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.341603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.341969 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.342146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.342329 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.342510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.342693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.342867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.342867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.343206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.343386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.343638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.343816 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.343993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.344183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 03 10:28:44.344364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.344364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.344364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.344364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.344364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.344364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.344364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.344364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.344364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.345848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.345848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.345848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.345848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.345848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.345848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.345848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.345848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.345848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.347307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.347307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.347307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.347307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.347307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.347307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.347307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.347307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.347307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.348787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.348787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.348787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.348787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.348787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.348787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.348787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.348787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.348787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.350497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.350497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.350497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.350497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.350497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.350497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.350497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.350497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.350497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.351948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.351948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.351948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.351948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.351948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.351948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.351948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.351948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.351948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.353394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.353394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.353394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.353394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.353394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.353394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.353394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.353394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.353394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.354926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.354926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.354926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.354926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.354926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.354926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.354926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.354926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] role.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.354926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] role.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.356395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.356395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.356395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.356395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.356395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.356395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.356395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.356395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.356395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.357812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.357812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.357812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.357812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.357812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.357812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.357812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.357812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.357812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.359273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.359273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.359273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.359273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.359273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.359273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.359273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.359273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.359273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.360720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.360720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.360720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.360720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.360720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.360720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.360720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.360720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.360720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.362127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.362127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.362127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.362127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.362127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.362127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.362127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.362127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.362127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.363578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.363578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.363578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.363578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.363578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.363578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.363578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.connection = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.363578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.363578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.365228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.365228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.365228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.365228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.365228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.365228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.365228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.365228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.365228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.366787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.366787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.366787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.366787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.366787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.366787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.366787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 10:28:44.366787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 03 10:28:44.398818 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.399077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.399378 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.408863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.431574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.431935 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.431935 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.440732 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.440976 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.441182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.460192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.460434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.498205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.500628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.554705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.555011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.557252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.557573 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.625663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.625956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.626269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.626556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.626855 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.627142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.627405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.627645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.627876 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.628103 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.628325 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.628549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.653137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.653411 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.653607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.653828 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.654037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.654255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.654472 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.655634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.655907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.656177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.656446 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.656710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.656961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.657239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.666884 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.669139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.681001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.681511 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.681511 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.682052 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.682052 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.682485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.682485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.682984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.682984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.682984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.683612 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.683612 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.684113 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.684329 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 10:28:44.700574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.700944 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 10:28:44.701180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.701397 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.701649 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 10:28:44.701853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 10:28:44.702053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.702255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.702498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 10:28:44.702702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.702702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.703077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.703275 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.703484 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.703484 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.703484 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.712239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.712858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.713061 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.713506 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.713886 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.714235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.714557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.714895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.720842 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.722455 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.724588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.724914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.725208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.725524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.725957 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.726226 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.726505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.726800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.727089 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.727381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.727656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.728725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.729077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.729409 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.729757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.730082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.730389 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.730703 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.731021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.731337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.731658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.731974 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.751952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.753683 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.753995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.757170 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.758886 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.759116 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.759321 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.759525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.759757 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.764322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.764672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.765000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.766278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.766590 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.771074 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.771432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.771628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.771921 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.772201 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.772482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.772689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.772895 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.773108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.773314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.777410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.777841 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.778188 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.778565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.778895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.779231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.779565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.779898 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.789915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.790277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.796031 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.797533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.797914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.799375 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.799692 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.802335 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.804226 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.805768 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.806126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.807702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.810910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.811231 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.811462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.811667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.811899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.812145 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.812432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.814677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.818739 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.819100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.819432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.819432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.820285 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.820488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.829538 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.829891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.830125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.830365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.838183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.838540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.838753 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.839079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.839305 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.840104 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.840384 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.843170 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.843466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.843766 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.844191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.844485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.846793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.848469 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.848791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.851798 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.852207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.852574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.852979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.853345 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.856139 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.856557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.856769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.857099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.857390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.857707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.857988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.866174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.866503 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.866845 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.867177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.867509 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.867837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.868178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.873943 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.876961 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.880516 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.882050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.883717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.883981 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.884278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.885375 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.888636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.890854 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.891814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.892386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.894642 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.896083 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.896396 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.896927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.897137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.897302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.897444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.903984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.905000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.905301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.905617 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.905851 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.908879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.909214 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.909591 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.909919 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.912776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.913071 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.913361 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.913675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.913966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.914251 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.914543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.914817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.915091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.915362 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.915660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.915925 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.916203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.916467 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.918952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.919254 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.919539 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.926524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 10:28:44.926842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.927089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.927376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.927662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.927934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.928207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.928486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.928876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.929164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.929449 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.929751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.930016 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.930285 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 10:28:44.952010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5569ebad59c0 pid: 80318 (default app) Aug 03 10:28:44.955742 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93c311-a2e7-4ace-8f7e-be25d089898a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:44.956096 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93c311-a2e7-4ace-8f7e-be25d089898a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:44.956329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93c311-a2e7-4ace-8f7e-be25d089898a None None] PATH_INFO: `/v3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:44.961496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1/1] 10.222.0.47 () {50 vars in 791 bytes} [Wed Aug 3 10:28:44 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:28:44.965649 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5569ebad59c0 pid: 80319 (default app) Aug 03 10:28:53.094867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6da6dc-ffe6-4d6c-8cfc-335e840d0a29 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:53.095463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6da6dc-ffe6-4d6c-8cfc-335e840d0a29 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:53.095818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6da6dc-ffe6-4d6c-8cfc-335e840d0a29 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:53.098336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 2/2] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:28:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:28:53.107008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac96fe-09e2-4d83-8470-4748877757a4 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:53.107541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac96fe-09e2-4d83-8470-4748877757a4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:53.107541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac96fe-09e2-4d83-8470-4748877757a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:53.183482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: INFO dbcounter [None req-1aac96fe-09e2-4d83-8470-4748877757a4 None None] Registered counter for database keystone Aug 03 10:28:53.206942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG oslo_db.sqlalchemy.engines [None req-1aac96fe-09e2-4d83-8470-4748877757a4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80319) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 03 10:28:53.214423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writer thread running {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 03 10:28:53.403908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG passlib.handlers.bcrypt [None req-1aac96fe-09e2-4d83-8470-4748877757a4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80319) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 03 10:28:53.404274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG passlib.handlers.bcrypt [None req-1aac96fe-09e2-4d83-8470-4748877757a4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80319) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 03 10:28:53.460939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-1aac96fe-09e2-4d83-8470-4748877757a4 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:28:53.462915 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1aac96fe-09e2-4d83-8470-4748877757a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:28:53.604944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1/3] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:28:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 501 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 03 10:28:53.701953 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6da6dc-ffe6-4d6c-8cfc-335e840d0a29 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:28:53.779152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: INFO dbcounter [None req-db6da6dc-ffe6-4d6c-8cfc-335e840d0a29 None None] Registered counter for database keystone Aug 03 10:28:53.802488 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG oslo_db.sqlalchemy.engines [None req-db6da6dc-ffe6-4d6c-8cfc-335e840d0a29 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80318) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 03 10:28:53.809106 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writer thread running {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 03 10:28:53.931892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6da6dc-ffe6-4d6c-8cfc-335e840d0a29 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:28:53.936413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f374653-a379-4768-8b57-2e493d290e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:28:53.937021 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f374653-a379-4768-8b57-2e493d290e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:53.937310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f374653-a379-4768-8b57-2e493d290e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:53.937624 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f374653-a379-4768-8b57-2e493d290e4f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:53.946781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f374653-a379-4768-8b57-2e493d290e4f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:28:54.002960 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2f374653-a379-4768-8b57-2e493d290e4f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 03 10:28:54.003304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f374653-a379-4768-8b57-2e493d290e4f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:28:54.014520 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 3/4] 10.222.0.47 () {58 vars in 1324 bytes} [Wed Aug 3 10:28:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 10:28:54.021655 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac96fe-09e2-4d83-8470-4748877757a4 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:28:54.037855 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac96fe-09e2-4d83-8470-4748877757a4 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:28:54.042647 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d149570-ce3b-4c04-a46b-f99f49f4756d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:28:54.043840 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d149570-ce3b-4c04-a46b-f99f49f4756d admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:54.044125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d149570-ce3b-4c04-a46b-f99f49f4756d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:54.044403 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d149570-ce3b-4c04-a46b-f99f49f4756d admin admin] PATH_INFO: `/v3/projects/a90579cdd4284a47979efcd0ca29e39a` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:54.046236 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d149570-ce3b-4c04-a46b-f99f49f4756d admin admin] RBAC: Authorizing `identity:get_project(project_id=a90579cdd4284a47979efcd0ca29e39a)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:28:54.097285 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3d149570-ce3b-4c04-a46b-f99f49f4756d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 03 10:28:54.097651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d149570-ce3b-4c04-a46b-f99f49f4756d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:28:54.103310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 2/5] 10.222.0.47 () {56 vars in 1207 bytes} [Wed Aug 3 10:28:54 2022] GET /identity/v3/projects/a90579cdd4284a47979efcd0ca29e39a => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:28:56.153143 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d2672-27e1-4005-84a8-6c6622c5af58 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:56.153766 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d2672-27e1-4005-84a8-6c6622c5af58 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:56.153766 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d2672-27e1-4005-84a8-6c6622c5af58 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:56.158017 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 4/6] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:28:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:28:56.164580 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befe596-5ab9-4770-a623-49e05cfb0c37 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:56.165092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befe596-5ab9-4770-a623-49e05cfb0c37 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:56.165208 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befe596-5ab9-4770-a623-49e05cfb0c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:56.186719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2befe596-5ab9-4770-a623-49e05cfb0c37 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:28:56.188284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2befe596-5ab9-4770-a623-49e05cfb0c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:28:56.196331 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 3/7] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:28:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 03 10:28:56.219140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d2672-27e1-4005-84a8-6c6622c5af58 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:28:56.232630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d2672-27e1-4005-84a8-6c6622c5af58 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:28:56.237171 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7f5c7-e3b3-4671-a7ae-ea511f7043dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:28:56.237830 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7f5c7-e3b3-4671-a7ae-ea511f7043dc admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:56.238113 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7f5c7-e3b3-4671-a7ae-ea511f7043dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:56.238389 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7f5c7-e3b3-4671-a7ae-ea511f7043dc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:56.247344 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7f5c7-e3b3-4671-a7ae-ea511f7043dc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:28:56.249139 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7f5c7-e3b3-4671-a7ae-ea511f7043dc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:28:56.259054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 5/8] 10.222.0.47 () {58 vars in 1324 bytes} [Wed Aug 3 10:28:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 10:28:56.264954 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befe596-5ab9-4770-a623-49e05cfb0c37 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:28:56.275144 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befe596-5ab9-4770-a623-49e05cfb0c37 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:28:56.280074 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cae76-c736-4fe2-882a-2e1dade0d626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:28:56.280801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cae76-c736-4fe2-882a-2e1dade0d626 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:56.281131 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cae76-c736-4fe2-882a-2e1dade0d626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:56.281455 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cae76-c736-4fe2-882a-2e1dade0d626 admin admin] PATH_INFO: `/v3/users/fb1a77890b95445982cf98d4ad77cb92` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:56.282980 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cae76-c736-4fe2-882a-2e1dade0d626 admin admin] RBAC: Authorizing `identity:get_user(user_id=fb1a77890b95445982cf98d4ad77cb92)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:28:56.284911 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cae76-c736-4fe2-882a-2e1dade0d626 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:28:56.287486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 4/9] 10.222.0.47 () {56 vars in 1198 bytes} [Wed Aug 3 10:28:56 2022] GET /identity/v3/users/fb1a77890b95445982cf98d4ad77cb92 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:28:58.414828 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b814c1c-1ba4-4d9a-bd21-ff0c2e1f0d7a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:58.415385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b814c1c-1ba4-4d9a-bd21-ff0c2e1f0d7a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:58.415732 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b814c1c-1ba4-4d9a-bd21-ff0c2e1f0d7a None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:58.418342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 6/10] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:28:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:28:58.425979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fe22d-7786-4845-bf5a-fffc05659bcd None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:58.427064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fe22d-7786-4845-bf5a-fffc05659bcd None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:58.427230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fe22d-7786-4845-bf5a-fffc05659bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:58.452629 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-563fe22d-7786-4845-bf5a-fffc05659bcd None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:28:58.454807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-563fe22d-7786-4845-bf5a-fffc05659bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:28:58.484913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70069f8-ecad-43e4-a93c-921d7df71406 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:58.485203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70069f8-ecad-43e4-a93c-921d7df71406 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:58.485518 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70069f8-ecad-43e4-a93c-921d7df71406 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:58.486921 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 7/11] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:28:58.492612 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8d6ae-3aa5-4f4d-8384-fef2bc9d08d2 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:58.492905 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8d6ae-3aa5-4f4d-8384-fef2bc9d08d2 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:58.493192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8d6ae-3aa5-4f4d-8384-fef2bc9d08d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:58.537883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG passlib.handlers.bcrypt [None req-92c8d6ae-3aa5-4f4d-8384-fef2bc9d08d2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80318) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 03 10:28:58.538309 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG passlib.handlers.bcrypt [None req-92c8d6ae-3aa5-4f4d-8384-fef2bc9d08d2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80318) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 03 10:28:58.583877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-92c8d6ae-3aa5-4f4d-8384-fef2bc9d08d2 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:28:58.585558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-92c8d6ae-3aa5-4f4d-8384-fef2bc9d08d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:28:58.600908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 5/12] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:28:58.641137 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fe22d-7786-4845-bf5a-fffc05659bcd None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:28:58.672337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:28:58.704383 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 8/13] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:28:58.734766 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8d6ae-3aa5-4f4d-8384-fef2bc9d08d2 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:28:58.764043 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fe22d-7786-4845-bf5a-fffc05659bcd None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:28:58.811366 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9f679-09d4-46ba-9c0e-604eac5fb863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:28:58.812231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9f679-09d4-46ba-9c0e-604eac5fb863 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:58.812739 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9f679-09d4-46ba-9c0e-604eac5fb863 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:58.813238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9f679-09d4-46ba-9c0e-604eac5fb863 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:58.835952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a9f679-09d4-46ba-9c0e-604eac5fb863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-c4a9f679-09d4-46ba-9c0e-604eac5fb863 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:28:58.852513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:28:58.853908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:28:58.855048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:28:58.855048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:28:58.855048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:28:58.855048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:28:58.855048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:28:58.855048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:28:58.855048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:28:58.855048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:28:58.855048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:28:58.855048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:28:58.855048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:28:58.855048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:28:58.855048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 6/14] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:28:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:28:58.865402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9f679-09d4-46ba-9c0e-604eac5fb863 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:28:58.871905 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8d6ae-3aa5-4f4d-8384-fef2bc9d08d2 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:28:58.874934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:28:58.930060 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a658694-d034-471f-83ee-b849f9803959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:28:58.931439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a658694-d034-471f-83ee-b849f9803959 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:58.931908 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a658694-d034-471f-83ee-b849f9803959 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:58.932331 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a658694-d034-471f-83ee-b849f9803959 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:59.006377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:28:59.032212 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9f679-09d4-46ba-9c0e-604eac5fb863 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:28:59.043218 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a658694-d034-471f-83ee-b849f9803959 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:28:59.045065 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a658694-d034-471f-83ee-b849f9803959 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:28:59.079610 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de5d58-83b8-4386-b5b0-8530e44d20ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:28:59.080486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de5d58-83b8-4386-b5b0-8530e44d20ff None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:59.080845 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de5d58-83b8-4386-b5b0-8530e44d20ff None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:59.081228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de5d58-83b8-4386-b5b0-8530e44d20ff None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:59.082223 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de5d58-83b8-4386-b5b0-8530e44d20ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:28:59.083230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de5d58-83b8-4386-b5b0-8530e44d20ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:28:59.084912 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de5d58-83b8-4386-b5b0-8530e44d20ff None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:28:59.107079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 7/15] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:28:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:28:59.112955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de5d58-83b8-4386-b5b0-8530e44d20ff None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:28:59.175168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 9/16] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:28:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 10:28:59.182053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a658694-d034-471f-83ee-b849f9803959 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:28:59.185459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:28:59.235823 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de5d58-83b8-4386-b5b0-8530e44d20ff None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:28:59.331936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a658694-d034-471f-83ee-b849f9803959 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:28:59.378985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002840e-c4e6-427a-9291-f511acd689ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:28:59.380493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002840e-c4e6-427a-9291-f511acd689ef None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:59.380775 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002840e-c4e6-427a-9291-f511acd689ef None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:59.381053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002840e-c4e6-427a-9291-f511acd689ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:59.398087 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f9e2b-8835-4833-8bc4-b1cd91bbf937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:28:59.400934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f9e2b-8835-4833-8bc4-b1cd91bbf937 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:59.401346 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f9e2b-8835-4833-8bc4-b1cd91bbf937 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:59.401751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f9e2b-8835-4833-8bc4-b1cd91bbf937 None admin] PATH_INFO: `/v3/users/fb1a77890b95445982cf98d4ad77cb92` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:59.404226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f9e2b-8835-4833-8bc4-b1cd91bbf937 None admin] RBAC: Authorizing `identity:get_user(user_id=fb1a77890b95445982cf98d4ad77cb92)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:28:59.406039 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f9e2b-8835-4833-8bc4-b1cd91bbf937 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:28:59.407768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002840e-c4e6-427a-9291-f511acd689ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:28:59.407999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 8/17] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:28:59 2022] GET /identity/v3/users/fb1a77890b95445982cf98d4ad77cb92 => generated 247 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:28:59.409289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002840e-c4e6-427a-9291-f511acd689ef None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:28:59.413989 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f9e2b-8835-4833-8bc4-b1cd91bbf937 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:28:59.430888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:28:59.441228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-1002840e-c4e6-427a-9291-f511acd689ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:28:59.441228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:28:59.441228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:28:59.441228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:28:59.441228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:28:59.441228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:28:59.441228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:28:59.441228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:28:59.441228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:28:59.441228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:28:59.441228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:28:59.441228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:28:59.441228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:28:59.441228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:28:59.441228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:28:59.443823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:28:59.444788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:28:59.444788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:28:59.444788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:28:59.444788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:28:59.444788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 10/18] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:28:59.452289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002840e-c4e6-427a-9291-f511acd689ef None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:28:59.522303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:28:59.533282 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f9e2b-8835-4833-8bc4-b1cd91bbf937 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:28:59.570117 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb89eb-deb3-4eb7-9f8f-70bf432b0cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:28:59.570788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb89eb-deb3-4eb7-9f8f-70bf432b0cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:59.571075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb89eb-deb3-4eb7-9f8f-70bf432b0cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:59.571358 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb89eb-deb3-4eb7-9f8f-70bf432b0cd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:59.572583 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb89eb-deb3-4eb7-9f8f-70bf432b0cd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:28:59.574093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb89eb-deb3-4eb7-9f8f-70bf432b0cd1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:28:59.575725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 9/19] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:28:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:28:59.580631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002840e-c4e6-427a-9291-f511acd689ef None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:28:59.581736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb89eb-deb3-4eb7-9f8f-70bf432b0cd1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:28:59.623890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439242d9-63c9-4e21-bf7a-7d4e217dff69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:28:59.624641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439242d9-63c9-4e21-bf7a-7d4e217dff69 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:59.625019 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439242d9-63c9-4e21-bf7a-7d4e217dff69 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:59.625499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439242d9-63c9-4e21-bf7a-7d4e217dff69 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:59.626435 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439242d9-63c9-4e21-bf7a-7d4e217dff69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:28:59.626785 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439242d9-63c9-4e21-bf7a-7d4e217dff69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:28:59.628518 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439242d9-63c9-4e21-bf7a-7d4e217dff69 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:28:59.650094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 11/20] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:28:59.693250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:28:59.720790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb89eb-deb3-4eb7-9f8f-70bf432b0cd1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:28:59.763587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67d0be-0747-4334-8777-1fe0dcb3315f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:28:59.764336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67d0be-0747-4334-8777-1fe0dcb3315f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:28:59.764689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67d0be-0747-4334-8777-1fe0dcb3315f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:28:59.765037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67d0be-0747-4334-8777-1fe0dcb3315f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:28:59.765990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67d0be-0747-4334-8777-1fe0dcb3315f None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.domain.id=default, user.id=fb1a77890b95445982cf98d4ad77cb92) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:28:59.766368 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67d0be-0747-4334-8777-1fe0dcb3315f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:28:59.767935 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67d0be-0747-4334-8777-1fe0dcb3315f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:28:59.776547 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 10/21] 10.222.0.47 () {56 vars in 1322 bytes} [Wed Aug 3 10:28:59 2022] GET /identity/v3/role_assignments?user.id=fb1a77890b95445982cf98d4ad77cb92&scope.domain.id=default&role.id=01714797cab143e1a7a150c068fa9315 => generated 233 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:02.538192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28856e3f-acfc-4ec7-a316-4bb8d753e905 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:02.538958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28856e3f-acfc-4ec7-a316-4bb8d753e905 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:02.538958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28856e3f-acfc-4ec7-a316-4bb8d753e905 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:02.540167 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 12/22] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:29:02.558598 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8e681-d4fa-4dea-a4ad-ef78f3d8b72e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:02.559387 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8e681-d4fa-4dea-a4ad-ef78f3d8b72e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:02.559387 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8e681-d4fa-4dea-a4ad-ef78f3d8b72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:02.587489 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-41c8e681-d4fa-4dea-a4ad-ef78f3d8b72e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:02.589015 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-41c8e681-d4fa-4dea-a4ad-ef78f3d8b72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:02.688838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 11/23] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:02.735101 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28856e3f-acfc-4ec7-a316-4bb8d753e905 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:02.884997 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28856e3f-acfc-4ec7-a316-4bb8d753e905 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:02.907318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:02.934874 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c238aec-a993-4b08-aee5-d0e4a7226b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:02.935521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c238aec-a993-4b08-aee5-d0e4a7226b77 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:02.935800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c238aec-a993-4b08-aee5-d0e4a7226b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:02.936070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c238aec-a993-4b08-aee5-d0e4a7226b77 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:02.936829 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c238aec-a993-4b08-aee5-d0e4a7226b77 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:02.938261 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c238aec-a993-4b08-aee5-d0e4a7226b77 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:03.087439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 13/24] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Aug 3 10:29:02 2022] POST /identity/v3/roles => generated 219 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:29:03.483428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c82183-891e-4a77-b42f-7eafb5617b5b None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:03.483940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c82183-891e-4a77-b42f-7eafb5617b5b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:03.484373 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c82183-891e-4a77-b42f-7eafb5617b5b None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:03.487427 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 12/25] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:29:03.509284 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f65b9-7e3f-44a5-a749-08319438a8ab None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:03.510142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f65b9-7e3f-44a5-a749-08319438a8ab None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:03.510645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f65b9-7e3f-44a5-a749-08319438a8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:03.555166 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1e8f65b9-7e3f-44a5-a749-08319438a8ab None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:03.557860 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1e8f65b9-7e3f-44a5-a749-08319438a8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:03.702290 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886d4aa-fec0-4264-88f7-6f99b032b48e None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:03.703726 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886d4aa-fec0-4264-88f7-6f99b032b48e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:03.704426 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886d4aa-fec0-4264-88f7-6f99b032b48e None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:03.705836 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 14/26] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:03.708057 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 13/27] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:29:03.722009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7483b-1efd-4517-b12c-6eacaed073e1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:03.722484 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7483b-1efd-4517-b12c-6eacaed073e1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:03.723036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7483b-1efd-4517-b12c-6eacaed073e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:03.779538 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-44a7483b-1efd-4517-b12c-6eacaed073e1 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:03.781382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-44a7483b-1efd-4517-b12c-6eacaed073e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:03.782284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd780db-cb94-4615-ab5f-8243dfa278e9 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:03.785587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd780db-cb94-4615-ab5f-8243dfa278e9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:03.786789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd780db-cb94-4615-ab5f-8243dfa278e9 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:03.789103 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 14/28] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:03.791690 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd780db-cb94-4615-ab5f-8243dfa278e9 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:03.900682 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:03.903609 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 15/29] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:03.905985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859fd89-1de1-49b3-9f94-87e9eaadf87d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:03.906577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859fd89-1de1-49b3-9f94-87e9eaadf87d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:03.907147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859fd89-1de1-49b3-9f94-87e9eaadf87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:03.931520 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd780db-cb94-4615-ab5f-8243dfa278e9 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:03.944333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0859fd89-1de1-49b3-9f94-87e9eaadf87d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:03.946868 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0859fd89-1de1-49b3-9f94-87e9eaadf87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:03.994038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:03.995152 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4de1e-a788-4fe5-9614-e7db7de17a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:03.996412 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4de1e-a788-4fe5-9614-e7db7de17a60 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:03.997042 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4de1e-a788-4fe5-9614-e7db7de17a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:03.997669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4de1e-a788-4fe5-9614-e7db7de17a60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:04.020235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4de1e-a788-4fe5-9614-e7db7de17a60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:04.023317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4de1e-a788-4fe5-9614-e7db7de17a60 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:04.056999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-57d4de1e-a788-4fe5-9614-e7db7de17a60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:29:04.056999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:04.056999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:04.056999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:04.056999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:04.056999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:04.056999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:04.056999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:04.056999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:04.056999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:04.056999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:04.056999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:04.056999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:29:04.056999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:29:04.056999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:04.058613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:04.060176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:29:04.060176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:29:04.060176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:29:04.060176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:04.060176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 15/30] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:04.065666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4de1e-a788-4fe5-9614-e7db7de17a60 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:04.191487 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4de1e-a788-4fe5-9614-e7db7de17a60 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:04.215026 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 16/31] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:04.217492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859fd89-1de1-49b3-9f94-87e9eaadf87d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:04.239975 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:04.266131 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee95655-c04b-4506-8194-b64704df8dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:04.267235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee95655-c04b-4506-8194-b64704df8dde None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:04.267590 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee95655-c04b-4506-8194-b64704df8dde None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:04.267993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee95655-c04b-4506-8194-b64704df8dde None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:04.269008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee95655-c04b-4506-8194-b64704df8dde None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:04.271210 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee95655-c04b-4506-8194-b64704df8dde None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:04.393359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859fd89-1de1-49b3-9f94-87e9eaadf87d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:04.449522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd46eb-945c-43c7-8b4b-605f476fc445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:04.450532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd46eb-945c-43c7-8b4b-605f476fc445 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:04.451004 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd46eb-945c-43c7-8b4b-605f476fc445 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:04.451469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd46eb-945c-43c7-8b4b-605f476fc445 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:04.452708 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd46eb-945c-43c7-8b4b-605f476fc445 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:04.453233 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd46eb-945c-43c7-8b4b-605f476fc445 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:04.455757 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd46eb-945c-43c7-8b4b-605f476fc445 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:04.476619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 17/32] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:04.478223 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd46eb-945c-43c7-8b4b-605f476fc445 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:04.484359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 16/33] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Aug 3 10:29:04 2022] POST /identity/v3/roles => generated 225 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:29:04.486837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c7f79-a0c6-4c0b-9882-7e0931f3d4a7 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:04.487342 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c7f79-a0c6-4c0b-9882-7e0931f3d4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:04.487813 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c7f79-a0c6-4c0b-9882-7e0931f3d4a7 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:04.490331 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 17/34] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:04.493823 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb5420-c741-448e-8fe6-01a80cff2ebd None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:04.493823 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb5420-c741-448e-8fe6-01a80cff2ebd None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:04.493823 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb5420-c741-448e-8fe6-01a80cff2ebd None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:04.497993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 18/35] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:04.497993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb5420-c741-448e-8fe6-01a80cff2ebd None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:04.523019 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:04.623751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb5420-c741-448e-8fe6-01a80cff2ebd None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:04.643329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd46eb-945c-43c7-8b4b-605f476fc445 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:04.671372 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5a918-d437-4b93-a296-75aa35833a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:04.672298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5a918-d437-4b93-a296-75aa35833a0a None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:04.672579 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5a918-d437-4b93-a296-75aa35833a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:04.672874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5a918-d437-4b93-a296-75aa35833a0a None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:04.673923 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5a918-d437-4b93-a296-75aa35833a0a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:04.675689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5a918-d437-4b93-a296-75aa35833a0a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:04.705514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be327c1a-c456-47de-ad5d-1f156c9e40a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:04.705514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be327c1a-c456-47de-ad5d-1f156c9e40a0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:04.705514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be327c1a-c456-47de-ad5d-1f156c9e40a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:04.705514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be327c1a-c456-47de-ad5d-1f156c9e40a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:04.717165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 19/36] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 3 10:29:04 2022] POST /identity/v3/projects => generated 301 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:29:04.719165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54843b9-eafd-4a90-8537-c9b89da8fb47 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:04.719664 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54843b9-eafd-4a90-8537-c9b89da8fb47 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:04.720180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54843b9-eafd-4a90-8537-c9b89da8fb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:04.720777 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be327c1a-c456-47de-ad5d-1f156c9e40a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-be327c1a-c456-47de-ad5d-1f156c9e40a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:04.727811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:04.729841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:04.731466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:04.731466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:04.731466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:04.731466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:04.731466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:04.731466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:04.731466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:04.731466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:04.731466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:04.731466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:04.731466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:04.731466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:04.731466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 18/37] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:29:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:04.731466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167fa5c-cbd0-490e-a45d-0abecf6fdd3a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:04.732634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167fa5c-cbd0-490e-a45d-0abecf6fdd3a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:04.732634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167fa5c-cbd0-490e-a45d-0abecf6fdd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:04.755347 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b54843b9-eafd-4a90-8537-c9b89da8fb47 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:04.757055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b54843b9-eafd-4a90-8537-c9b89da8fb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:04.759008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7167fa5c-cbd0-490e-a45d-0abecf6fdd3a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:04.765091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7167fa5c-cbd0-490e-a45d-0abecf6fdd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:04.809177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:04.867199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 20/38] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:04.868474 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54843b9-eafd-4a90-8537-c9b89da8fb47 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:04.897793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 19/39] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:04.899705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167fa5c-cbd0-490e-a45d-0abecf6fdd3a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:04.933238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:05.040985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54843b9-eafd-4a90-8537-c9b89da8fb47 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:05.047724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167fa5c-cbd0-490e-a45d-0abecf6fdd3a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:05.083602 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4ce9e-2c06-4ada-8440-780cf14c0bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:05.084383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4ce9e-2c06-4ada-8440-780cf14c0bef None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:05.084762 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4ce9e-2c06-4ada-8440-780cf14c0bef None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:05.085131 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4ce9e-2c06-4ada-8440-780cf14c0bef None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:05.086030 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4ce9e-2c06-4ada-8440-780cf14c0bef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:05.086459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200163a4-7902-407f-a32e-5a9c330890d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:05.086636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4ce9e-2c06-4ada-8440-780cf14c0bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:05.087095 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200163a4-7902-407f-a32e-5a9c330890d3 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:05.087375 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200163a4-7902-407f-a32e-5a9c330890d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:05.087657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200163a4-7902-407f-a32e-5a9c330890d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:05.088638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4ce9e-2c06-4ada-8440-780cf14c0bef None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:05.088795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200163a4-7902-407f-a32e-5a9c330890d3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:05.090068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200163a4-7902-407f-a32e-5a9c330890d3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:05.104830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 21/40] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:05.105924 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4ce9e-2c06-4ada-8440-780cf14c0bef None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:05.111548 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 20/41] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Aug 3 10:29:04 2022] POST /identity/v3/roles => generated 223 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:29:05.114221 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200163a4-7902-407f-a32e-5a9c330890d3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:05.155700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:05.218814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:05.250782 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200163a4-7902-407f-a32e-5a9c330890d3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:05.285729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4ce9e-2c06-4ada-8440-780cf14c0bef None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:05.313068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02398f9f-edc8-4533-a93d-0a8da3692814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:05.313879 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02398f9f-edc8-4533-a93d-0a8da3692814 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:05.314168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02398f9f-edc8-4533-a93d-0a8da3692814 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:05.314454 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02398f9f-edc8-4533-a93d-0a8da3692814 None admin] PATH_INFO: `/v3/users/fb1a77890b95445982cf98d4ad77cb92` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:05.316245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02398f9f-edc8-4533-a93d-0a8da3692814 None admin] RBAC: Authorizing `identity:get_user(user_id=fb1a77890b95445982cf98d4ad77cb92)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:05.318446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02398f9f-edc8-4533-a93d-0a8da3692814 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:05.320290 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 21/42] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:05 2022] GET /identity/v3/users/fb1a77890b95445982cf98d4ad77cb92 => generated 247 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:05.326353 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02398f9f-edc8-4533-a93d-0a8da3692814 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:05.336125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202242de-6a65-4589-b4b9-ece1ca73bcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:05.337578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202242de-6a65-4589-b4b9-ece1ca73bcba None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:05.338472 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202242de-6a65-4589-b4b9-ece1ca73bcba None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:05.339098 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202242de-6a65-4589-b4b9-ece1ca73bcba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:05.341219 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202242de-6a65-4589-b4b9-ece1ca73bcba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:05.343919 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202242de-6a65-4589-b4b9-ece1ca73bcba None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:05.347148 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 22/43] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:29:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:05.353312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202242de-6a65-4589-b4b9-ece1ca73bcba None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:05.444644 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02398f9f-edc8-4533-a93d-0a8da3692814 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:05.465028 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:05.470777 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202242de-6a65-4589-b4b9-ece1ca73bcba None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:05.485117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6d411-d59e-47ad-a502-30194a9a6880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:05.485817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6d411-d59e-47ad-a502-30194a9a6880 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:05.486103 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6d411-d59e-47ad-a502-30194a9a6880 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:05.486372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6d411-d59e-47ad-a502-30194a9a6880 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:05.487585 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f6d411-d59e-47ad-a502-30194a9a6880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:05.489656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f6d411-d59e-47ad-a502-30194a9a6880 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:05.491338 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 22/44] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:29:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:05.497972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6d411-d59e-47ad-a502-30194a9a6880 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:05.511661 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34446f-2198-41d7-8bec-048b116fb570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:05.512307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34446f-2198-41d7-8bec-048b116fb570 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:05.512602 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34446f-2198-41d7-8bec-048b116fb570 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:05.512882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34446f-2198-41d7-8bec-048b116fb570 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:05.513634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34446f-2198-41d7-8bec-048b116fb570 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:05.514940 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:05.515095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34446f-2198-41d7-8bec-048b116fb570 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:05.544359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 23/45] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 3 10:29:05 2022] POST /identity/v3/projects => generated 312 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:29:05.627913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6d411-d59e-47ad-a502-30194a9a6880 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:05.692615 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f133092-5a8b-47e4-9af8-47af3a0fd282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:05.693858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f133092-5a8b-47e4-9af8-47af3a0fd282 None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:05.694366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f133092-5a8b-47e4-9af8-47af3a0fd282 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:05.694890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f133092-5a8b-47e4-9af8-47af3a0fd282 None admin] PATH_INFO: `/v3/domains/default/users/fb1a77890b95445982cf98d4ad77cb92/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:05.698701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f133092-5a8b-47e4-9af8-47af3a0fd282 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=fb1a77890b95445982cf98d4ad77cb92, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:05.701510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f133092-5a8b-47e4-9af8-47af3a0fd282 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:05.724673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 23/46] 10.222.0.47 () {58 vars in 1361 bytes} [Wed Aug 3 10:29:05 2022] PUT /identity/v3/domains/default/users/fb1a77890b95445982cf98d4ad77cb92/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:29:07.627638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9a87e-a7f3-4950-89f7-5ceef8d8caa1 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:07.628244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9a87e-a7f3-4950-89f7-5ceef8d8caa1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:07.628595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9a87e-a7f3-4950-89f7-5ceef8d8caa1 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:07.631017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 24/47] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:07.637009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73626b3b-a02d-4e41-93bd-28ae0bb5b772 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:07.637774 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73626b3b-a02d-4e41-93bd-28ae0bb5b772 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:07.637952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73626b3b-a02d-4e41-93bd-28ae0bb5b772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:07.658870 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-73626b3b-a02d-4e41-93bd-28ae0bb5b772 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:07.660086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-73626b3b-a02d-4e41-93bd-28ae0bb5b772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:07.754564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 24/48] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:07.778358 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9a87e-a7f3-4950-89f7-5ceef8d8caa1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:07.885291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c447c5c-778d-4d07-9fdf-b416d20f9c33 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:07.885742 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c447c5c-778d-4d07-9fdf-b416d20f9c33 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:07.886221 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c447c5c-778d-4d07-9fdf-b416d20f9c33 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:07.888665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 25/49] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:07.898121 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6ba8e-f6b4-4b97-b4c0-bd251d39389c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:07.898630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6ba8e-f6b4-4b97-b4c0-bd251d39389c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:07.899127 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6ba8e-f6b4-4b97-b4c0-bd251d39389c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:07.909657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:07.924461 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3ca6ba8e-f6b4-4b97-b4c0-bd251d39389c None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:07.926085 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3ca6ba8e-f6b4-4b97-b4c0-bd251d39389c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:07.941001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9a87e-a7f3-4950-89f7-5ceef8d8caa1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:07.993947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9c54b-1a0e-444c-b434-5135d52e45fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:07.994619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9c54b-1a0e-444c-b434-5135d52e45fd None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:07.994907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9c54b-1a0e-444c-b434-5135d52e45fd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:07.995188 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9c54b-1a0e-444c-b434-5135d52e45fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:07.996399 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9c54b-1a0e-444c-b434-5135d52e45fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:07.997918 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9c54b-1a0e-444c-b434-5135d52e45fd None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:07.999600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 25/50] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:29:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:08.006238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9c54b-1a0e-444c-b434-5135d52e45fd None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:08.008038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 26/51] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:08.028142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:08.035145 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6ba8e-f6b4-4b97-b4c0-bd251d39389c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:08.132362 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9c54b-1a0e-444c-b434-5135d52e45fd None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:08.171865 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6ba8e-f6b4-4b97-b4c0-bd251d39389c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:08.174971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34775b57-328e-483e-8149-ab7241abf6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:08.175971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34775b57-328e-483e-8149-ab7241abf6f4 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:08.176460 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34775b57-328e-483e-8149-ab7241abf6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:08.176886 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34775b57-328e-483e-8149-ab7241abf6f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:08.178125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34775b57-328e-483e-8149-ab7241abf6f4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:08.179920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34775b57-328e-483e-8149-ab7241abf6f4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:08.209990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:08.210502 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 26/52] 10.222.0.47 () {60 vars in 1140 bytes} [Wed Aug 3 10:29:08 2022] POST /identity/v3/projects => generated 298 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:29:08.217454 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edeeef3-8709-4586-8940-985f0bae1a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:08.218258 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edeeef3-8709-4586-8940-985f0bae1a57 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:08.218606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edeeef3-8709-4586-8940-985f0bae1a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:08.219029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edeeef3-8709-4586-8940-985f0bae1a57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:08.233101 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edeeef3-8709-4586-8940-985f0bae1a57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-9edeeef3-8709-4586-8940-985f0bae1a57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:08.239215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:08.240948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:08.242404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:08.242404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:08.242404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:08.242404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:08.242404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:08.242404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:08.242404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:08.242404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:08.242404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:08.242404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:08.242404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:08.242404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:08.242404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 27/53] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:29:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:29:08.249034 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34775b57-328e-483e-8149-ab7241abf6f4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:08.345875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:08.394434 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34775b57-328e-483e-8149-ab7241abf6f4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:08.457205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cada7-0567-492c-8e1c-1edd7c94ace9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:08.458338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cada7-0567-492c-8e1c-1edd7c94ace9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:08.458856 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cada7-0567-492c-8e1c-1edd7c94ace9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:08.459349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cada7-0567-492c-8e1c-1edd7c94ace9 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:08.460749 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80cada7-0567-492c-8e1c-1edd7c94ace9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:08.461305 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80cada7-0567-492c-8e1c-1edd7c94ace9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:08.464276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80cada7-0567-492c-8e1c-1edd7c94ace9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:08.485534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 27/54] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:08.489446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edeeef3-8709-4586-8940-985f0bae1a57 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:08.615284 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edeeef3-8709-4586-8940-985f0bae1a57 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:08.657628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70367d-ac05-41a9-b40f-66b869218843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:08.658608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70367d-ac05-41a9-b40f-66b869218843 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:08.658608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70367d-ac05-41a9-b40f-66b869218843 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:08.658979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70367d-ac05-41a9-b40f-66b869218843 None admin] PATH_INFO: `/v3/users/fb1a77890b95445982cf98d4ad77cb92` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:08.660894 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70367d-ac05-41a9-b40f-66b869218843 None admin] RBAC: Authorizing `identity:get_user(user_id=fb1a77890b95445982cf98d4ad77cb92)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:08.662525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70367d-ac05-41a9-b40f-66b869218843 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:08.664332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 28/55] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:08 2022] GET /identity/v3/users/fb1a77890b95445982cf98d4ad77cb92 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:08.672955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cada7-0567-492c-8e1c-1edd7c94ace9 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:08.795443 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cada7-0567-492c-8e1c-1edd7c94ace9 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:08.830183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:08.837160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfb2e8-8ed2-44c0-a5b4-a2823b30b1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:08.838194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfb2e8-8ed2-44c0-a5b4-a2823b30b1df None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:08.838540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfb2e8-8ed2-44c0-a5b4-a2823b30b1df None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:08.838880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfb2e8-8ed2-44c0-a5b4-a2823b30b1df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:08.840178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfb2e8-8ed2-44c0-a5b4-a2823b30b1df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:08.841784 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfb2e8-8ed2-44c0-a5b4-a2823b30b1df None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:08.843516 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 28/56] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:29:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:08.852573 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70367d-ac05-41a9-b40f-66b869218843 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:08.922958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:08.974489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70367d-ac05-41a9-b40f-66b869218843 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:09.015926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38d6ae-608b-4f90-b6fe-6e59421e0f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:09.016721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38d6ae-608b-4f90-b6fe-6e59421e0f05 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:09.017107 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38d6ae-608b-4f90-b6fe-6e59421e0f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:09.017509 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38d6ae-608b-4f90-b6fe-6e59421e0f05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:09.018583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38d6ae-608b-4f90-b6fe-6e59421e0f05 None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.domain.id=default, user.id=fb1a77890b95445982cf98d4ad77cb92) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:09.018948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38d6ae-608b-4f90-b6fe-6e59421e0f05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:09.020608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38d6ae-608b-4f90-b6fe-6e59421e0f05 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:09.030256 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 29/57] 10.222.0.47 () {56 vars in 1322 bytes} [Wed Aug 3 10:29:08 2022] GET /identity/v3/role_assignments?user.id=fb1a77890b95445982cf98d4ad77cb92&scope.domain.id=default&role.id=01714797cab143e1a7a150c068fa9315 => generated 530 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:10.329742 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae962d-0d17-4786-8acd-a81154504300 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:10.330290 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae962d-0d17-4786-8acd-a81154504300 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:10.330408 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae962d-0d17-4786-8acd-a81154504300 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:10.331785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 29/58] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:10.337459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13760db-687a-402a-8b9c-8a51ba0d3796 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:10.338241 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13760db-687a-402a-8b9c-8a51ba0d3796 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:10.338422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13760db-687a-402a-8b9c-8a51ba0d3796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:10.357874 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e13760db-687a-402a-8b9c-8a51ba0d3796 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:10.359190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e13760db-687a-402a-8b9c-8a51ba0d3796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:10.431674 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 30/59] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:10.455928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae962d-0d17-4786-8acd-a81154504300 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:10.564040 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae962d-0d17-4786-8acd-a81154504300 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:10.600778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6acc3f-8844-4d18-91ea-c1f49c462077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:10.601361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6acc3f-8844-4d18-91ea-c1f49c462077 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:10.601687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6acc3f-8844-4d18-91ea-c1f49c462077 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:10.601977 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6acc3f-8844-4d18-91ea-c1f49c462077 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:10.603216 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6acc3f-8844-4d18-91ea-c1f49c462077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:10.604681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6acc3f-8844-4d18-91ea-c1f49c462077 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:10.606376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 30/60] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:29:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:10.612339 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13760db-687a-402a-8b9c-8a51ba0d3796 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:10.675382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:10.730781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13760db-687a-402a-8b9c-8a51ba0d3796 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:10.768809 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e75bf-5e55-419d-815c-cfdc62bff0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:10.769537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e75bf-5e55-419d-815c-cfdc62bff0e9 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:10.770395 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e75bf-5e55-419d-815c-cfdc62bff0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:10.770718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e75bf-5e55-419d-815c-cfdc62bff0e9 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:10.771501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e75bf-5e55-419d-815c-cfdc62bff0e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:10.773091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e75bf-5e55-419d-815c-cfdc62bff0e9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:10.805262 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 31/61] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:29:10 2022] POST /identity/v3/users => generated 275 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:29:14.235252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbd719-8ebd-4d49-afb5-f10baceb4f8f None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:14.236022 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbd719-8ebd-4d49-afb5-f10baceb4f8f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:14.236232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbd719-8ebd-4d49-afb5-f10baceb4f8f None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:14.239164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 31/62] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:14.247270 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5df63-7db2-4c14-b130-53918195aa6e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:14.248000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5df63-7db2-4c14-b130-53918195aa6e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:14.248000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5df63-7db2-4c14-b130-53918195aa6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:14.274634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-dbb5df63-7db2-4c14-b130-53918195aa6e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:14.275888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-dbb5df63-7db2-4c14-b130-53918195aa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:14.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 32/63] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:14.450650 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbd719-8ebd-4d49-afb5-f10baceb4f8f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:14.543927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:14.601774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbd719-8ebd-4d49-afb5-f10baceb4f8f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:14.684443 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e27c4-a64e-43a5-a8e0-c9d8b1184cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:14.685845 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e27c4-a64e-43a5-a8e0-c9d8b1184cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:14.686347 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e27c4-a64e-43a5-a8e0-c9d8b1184cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:14.686960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e27c4-a64e-43a5-a8e0-c9d8b1184cf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:14.713657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e27c4-a64e-43a5-a8e0-c9d8b1184cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-cc2e27c4-a64e-43a5-a8e0-c9d8b1184cf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:14.718999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:14.721121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:14.723209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:14.723209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:14.723209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:14.723209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:14.723209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:14.723209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:14.723209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:14.723209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:14.723209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:14.723209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:14.723209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:14.723209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:14.723209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 32/64] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:14.727118 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5df63-7db2-4c14-b130-53918195aa6e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:14.850302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:14.938642 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db637190-0ae7-459a-b161-ee7e3603d948 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:14.939221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db637190-0ae7-459a-b161-ee7e3603d948 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:14.939657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db637190-0ae7-459a-b161-ee7e3603d948 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:14.941294 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 33/65] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:14.947638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234e82f-8643-46c5-aed6-661a40743354 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:14.948216 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234e82f-8643-46c5-aed6-661a40743354 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:14.949240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234e82f-8643-46c5-aed6-661a40743354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:14.988540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5df63-7db2-4c14-b130-53918195aa6e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:14.999335 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b234e82f-8643-46c5-aed6-661a40743354 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:15.001803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b234e82f-8643-46c5-aed6-661a40743354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:15.041502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc4853-cc7b-4c67-bf47-ae427142d76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:15.043098 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc4853-cc7b-4c67-bf47-ae427142d76f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:15.044037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc4853-cc7b-4c67-bf47-ae427142d76f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:15.046211 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc4853-cc7b-4c67-bf47-ae427142d76f None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:15.048185 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc4853-cc7b-4c67-bf47-ae427142d76f None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:15.051723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc4853-cc7b-4c67-bf47-ae427142d76f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:15.054781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc4853-cc7b-4c67-bf47-ae427142d76f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:15.107286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 33/66] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:15.115058 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc4853-cc7b-4c67-bf47-ae427142d76f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:15.265896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 34/67] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:15.269983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909ea45-e017-4e24-b9c6-4d8970505ff8 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:15.270679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909ea45-e017-4e24-b9c6-4d8970505ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:15.271458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909ea45-e017-4e24-b9c6-4d8970505ff8 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:15.276106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 35/68] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:15.282628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba038bc-387a-4d2a-bddc-83887754ab24 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:15.285277 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba038bc-387a-4d2a-bddc-83887754ab24 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:15.286582 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba038bc-387a-4d2a-bddc-83887754ab24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:15.330683 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7ba038bc-387a-4d2a-bddc-83887754ab24 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:15.334359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7ba038bc-387a-4d2a-bddc-83887754ab24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:15.344566 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc4853-cc7b-4c67-bf47-ae427142d76f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:15.346405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:15.384906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd579d50-75b3-4110-90b2-02b176678bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:15.385915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd579d50-75b3-4110-90b2-02b176678bfb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:15.386379 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd579d50-75b3-4110-90b2-02b176678bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:15.386904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd579d50-75b3-4110-90b2-02b176678bfb None admin] PATH_INFO: `/v3/users/3829732fccee407fa473d0c55a5aa1d1` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:15.405034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd579d50-75b3-4110-90b2-02b176678bfb None admin] RBAC: Authorizing `identity:get_user(user_id=3829732fccee407fa473d0c55a5aa1d1)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:15.406930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd579d50-75b3-4110-90b2-02b176678bfb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:15.408976 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 34/69] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:15 2022] GET /identity/v3/users/3829732fccee407fa473d0c55a5aa1d1 => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:15.410167 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd579d50-75b3-4110-90b2-02b176678bfb None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:15.485099 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:15.505180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 36/70] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:15.514772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba038bc-387a-4d2a-bddc-83887754ab24 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:15.605220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd579d50-75b3-4110-90b2-02b176678bfb None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:15.651345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a1ea3-e98b-42d9-9ccc-28753a76a2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:15.651955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a1ea3-e98b-42d9-9ccc-28753a76a2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:15.652607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a1ea3-e98b-42d9-9ccc-28753a76a2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:15.653088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a1ea3-e98b-42d9-9ccc-28753a76a2f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:15.664601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a1ea3-e98b-42d9-9ccc-28753a76a2f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-d34a1ea3-e98b-42d9-9ccc-28753a76a2f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:15.669807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:15.670866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:15.672321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:15.672321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:15.672321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:15.672321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:15.672321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:15.672321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:15.672321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:15.672321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:15.672321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:15.672321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:15.672321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:15.672321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:15.672321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 35/71] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:29:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:15.672321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a1ea3-e98b-42d9-9ccc-28753a76a2f4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:15.703278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:15.743667 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba038bc-387a-4d2a-bddc-83887754ab24 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:15.798896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95af7a-cf81-41ef-839f-c59e438aa36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:15.799551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95af7a-cf81-41ef-839f-c59e438aa36f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:15.799841 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95af7a-cf81-41ef-839f-c59e438aa36f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:15.800126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95af7a-cf81-41ef-839f-c59e438aa36f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:15.819423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a1ea3-e98b-42d9-9ccc-28753a76a2f4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:15.872136 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:15.888285 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95af7a-cf81-41ef-839f-c59e438aa36f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:15.889952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95af7a-cf81-41ef-839f-c59e438aa36f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:15.891571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817fe3e-9370-4e86-bb1a-9d1048d144b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:15.892492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817fe3e-9370-4e86-bb1a-9d1048d144b0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:15.892896 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817fe3e-9370-4e86-bb1a-9d1048d144b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:15.893317 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817fe3e-9370-4e86-bb1a-9d1048d144b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:15.914212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817fe3e-9370-4e86-bb1a-9d1048d144b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-d817fe3e-9370-4e86-bb1a-9d1048d144b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:15.924161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:15.925530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:15.926546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:15.926546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:15.926546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:15.926546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:15.926546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:15.926546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:15.926546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:15.926546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:15.926546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:15.926546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:15.926546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:29:15.926546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:15.926546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 36/72] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:29:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:15.928635 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5aad9-1203-4c3f-bcbe-5baf0fece38e None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:15.929118 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5aad9-1203-4c3f-bcbe-5baf0fece38e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:15.929621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5aad9-1203-4c3f-bcbe-5baf0fece38e None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:15.931908 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 37/73] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:15.935871 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edc0f1-3593-4305-bd07-7587664a1d19 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:15.935871 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edc0f1-3593-4305-bd07-7587664a1d19 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:15.935871 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edc0f1-3593-4305-bd07-7587664a1d19 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:15.943194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 38/74] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:15.943194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edc0f1-3593-4305-bd07-7587664a1d19 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:16.030583 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 37/75] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:29:16.036236 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95af7a-cf81-41ef-839f-c59e438aa36f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:16.088406 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:16.109804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edc0f1-3593-4305-bd07-7587664a1d19 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:16.154217 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f7453-9e7f-4271-bdc4-74223448cefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:16.154854 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f7453-9e7f-4271-bdc4-74223448cefd None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:16.155142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f7453-9e7f-4271-bdc4-74223448cefd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:16.155424 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f7453-9e7f-4271-bdc4-74223448cefd None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:16.156179 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f7453-9e7f-4271-bdc4-74223448cefd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:16.156484 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f7453-9e7f-4271-bdc4-74223448cefd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:16.157938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f7453-9e7f-4271-bdc4-74223448cefd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:16.158374 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:16.167268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 39/76] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:16.168635 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b7b48-f271-420c-96de-d4afc7e27c27 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:16.168776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b7b48-f271-420c-96de-d4afc7e27c27 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:16.169024 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b7b48-f271-420c-96de-d4afc7e27c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:16.174787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95af7a-cf81-41ef-839f-c59e438aa36f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:16.197680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-457b7b48-f271-420c-96de-d4afc7e27c27 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:16.199804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-457b7b48-f271-420c-96de-d4afc7e27c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:16.212378 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07556c62-29d2-426c-ae84-0e3556970007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:16.213278 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07556c62-29d2-426c-ae84-0e3556970007 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:16.213607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07556c62-29d2-426c-ae84-0e3556970007 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:16.213892 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07556c62-29d2-426c-ae84-0e3556970007 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:16.214633 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07556c62-29d2-426c-ae84-0e3556970007 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:16.214944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07556c62-29d2-426c-ae84-0e3556970007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:16.216327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07556c62-29d2-426c-ae84-0e3556970007 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:16.225385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 38/77] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:29:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:16.226711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b43e5-4b6e-411f-a1ee-6d74a8f54599 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:16.227004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b43e5-4b6e-411f-a1ee-6d74a8f54599 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:16.227290 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b43e5-4b6e-411f-a1ee-6d74a8f54599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:16.256364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-099b43e5-4b6e-411f-a1ee-6d74a8f54599 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:16.258625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-099b43e5-4b6e-411f-a1ee-6d74a8f54599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:16.291871 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 40/78] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:16.292989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b7b48-f271-420c-96de-d4afc7e27c27 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:16.370655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:16.376052 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 39/79] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:16.377594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b43e5-4b6e-411f-a1ee-6d74a8f54599 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:16.425384 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b7b48-f271-420c-96de-d4afc7e27c27 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:16.478257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd47465-2fbe-454a-aa29-1243ea364d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:16.479089 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd47465-2fbe-454a-aa29-1243ea364d7c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:16.479508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd47465-2fbe-454a-aa29-1243ea364d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:16.479901 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd47465-2fbe-454a-aa29-1243ea364d7c None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:16.481279 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd47465-2fbe-454a-aa29-1243ea364d7c None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:16.483048 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd47465-2fbe-454a-aa29-1243ea364d7c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:16.485023 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 41/80] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:16 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:16.486619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd47465-2fbe-454a-aa29-1243ea364d7c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:16.509021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:16.534780 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b43e5-4b6e-411f-a1ee-6d74a8f54599 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:16.582692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134dcd8-4881-40d8-8b48-b61446612bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:16.583524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134dcd8-4881-40d8-8b48-b61446612bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:16.583946 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134dcd8-4881-40d8-8b48-b61446612bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:16.584296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134dcd8-4881-40d8-8b48-b61446612bd0 None admin] PATH_INFO: `/v3/users/fb1a77890b95445982cf98d4ad77cb92` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:16.585871 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134dcd8-4881-40d8-8b48-b61446612bd0 None admin] RBAC: Authorizing `identity:get_user(user_id=fb1a77890b95445982cf98d4ad77cb92)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:16.588432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134dcd8-4881-40d8-8b48-b61446612bd0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:16.590340 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 40/81] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:16 2022] GET /identity/v3/users/fb1a77890b95445982cf98d4ad77cb92 => generated 247 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:16.591665 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134dcd8-4881-40d8-8b48-b61446612bd0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:16.630098 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd47465-2fbe-454a-aa29-1243ea364d7c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:16.683399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ab629-2604-4954-9a14-c172e169b8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:16.684445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ab629-2604-4954-9a14-c172e169b8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:16.684692 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ab629-2604-4954-9a14-c172e169b8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:16.685147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ab629-2604-4954-9a14-c172e169b8c0 None admin] PATH_INFO: `/v3/users/3829732fccee407fa473d0c55a5aa1d1` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:16.687148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ab629-2604-4954-9a14-c172e169b8c0 None admin] RBAC: Authorizing `identity:get_user(user_id=3829732fccee407fa473d0c55a5aa1d1)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:16.689702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ab629-2604-4954-9a14-c172e169b8c0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:16.693077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 42/82] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:16 2022] GET /identity/v3/users/3829732fccee407fa473d0c55a5aa1d1 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:16.694472 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ab629-2604-4954-9a14-c172e169b8c0 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:16.711415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:16.752734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134dcd8-4881-40d8-8b48-b61446612bd0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:16.807946 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a74aa1-0978-4a0a-9db3-eaa523600ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:16.808966 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a74aa1-0978-4a0a-9db3-eaa523600ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:16.809426 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a74aa1-0978-4a0a-9db3-eaa523600ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:16.809888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a74aa1-0978-4a0a-9db3-eaa523600ca1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:16.811670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a74aa1-0978-4a0a-9db3-eaa523600ca1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:16.814076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a74aa1-0978-4a0a-9db3-eaa523600ca1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:16.817442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 41/83] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:29:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:16.818464 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a74aa1-0978-4a0a-9db3-eaa523600ca1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:16.842444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:16.846982 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ab629-2604-4954-9a14-c172e169b8c0 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:16.907502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9920965-e656-45ea-9938-c30d7222361c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:16.908626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9920965-e656-45ea-9938-c30d7222361c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:16.909135 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9920965-e656-45ea-9938-c30d7222361c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:16.909669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9920965-e656-45ea-9938-c30d7222361c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:16.929572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9920965-e656-45ea-9938-c30d7222361c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:16.939027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a74aa1-0978-4a0a-9db3-eaa523600ca1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-d9920965-e656-45ea-9938-c30d7222361c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:16.939276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:16.940767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:16.942140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:16.942140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:16.942140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:16.942140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:16.942140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:16.942140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:16.942140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:16.942140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:16.942140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:16.942140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:16.942140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:16.942140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:16.942140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 43/84] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:16.944444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9920965-e656-45ea-9938-c30d7222361c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:16.980843 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236a8a2-0f1a-457f-9cc5-eb71f1741ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:16.981505 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236a8a2-0f1a-457f-9cc5-eb71f1741ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:16.981790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236a8a2-0f1a-457f-9cc5-eb71f1741ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:16.982072 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236a8a2-0f1a-457f-9cc5-eb71f1741ca6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:16.982884 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236a8a2-0f1a-457f-9cc5-eb71f1741ca6 None admin] RBAC: Adding query filter params (role.id=82be5cfedcb3407f8d14522ef87247ac, scope.project.id=998a52ff5527415da541925c79fddaab, user.id=3829732fccee407fa473d0c55a5aa1d1) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:16.983199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236a8a2-0f1a-457f-9cc5-eb71f1741ca6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:16.984685 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236a8a2-0f1a-457f-9cc5-eb71f1741ca6 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:16.990981 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 42/85] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:16 2022] GET /identity/v3/role_assignments?user.id=3829732fccee407fa473d0c55a5aa1d1&scope.project.id=998a52ff5527415da541925c79fddaab&role.id=82be5cfedcb3407f8d14522ef87247ac => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:16.991847 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236a8a2-0f1a-457f-9cc5-eb71f1741ca6 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:17.059780 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:17.129796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9920965-e656-45ea-9938-c30d7222361c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:17.143618 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:17.172705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236a8a2-0f1a-457f-9cc5-eb71f1741ca6 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:17.199233 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e38eb-6656-4d64-9552-9ad44aae27c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:17.200423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e38eb-6656-4d64-9552-9ad44aae27c8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:17.200718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e38eb-6656-4d64-9552-9ad44aae27c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:17.200995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e38eb-6656-4d64-9552-9ad44aae27c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:17.231534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53acac2-0722-42df-ac53-544b9899d6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:17.232584 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53acac2-0722-42df-ac53-544b9899d6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:17.232998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53acac2-0722-42df-ac53-544b9899d6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:17.233436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53acac2-0722-42df-ac53-544b9899d6d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:17.329745 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53acac2-0722-42df-ac53-544b9899d6d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:17.331370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e38eb-6656-4d64-9552-9ad44aae27c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:17.332696 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53acac2-0722-42df-ac53-544b9899d6d4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:17.333406 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e38eb-6656-4d64-9552-9ad44aae27c8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:17.450160 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:17.451584 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:17.484199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 43/86] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:29:17.485181 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53acac2-0722-42df-ac53-544b9899d6d4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:17.510809 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 44/87] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:29:17.515109 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e38eb-6656-4d64-9552-9ad44aae27c8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:17.652899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53acac2-0722-42df-ac53-544b9899d6d4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:17.673662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e38eb-6656-4d64-9552-9ad44aae27c8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:17.702348 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e29fc-bc19-4a00-ade8-a3ddc6084bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:17.702995 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e29fc-bc19-4a00-ade8-a3ddc6084bcf None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:17.703293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e29fc-bc19-4a00-ade8-a3ddc6084bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:17.703578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e29fc-bc19-4a00-ade8-a3ddc6084bcf None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:17.704286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e29fc-bc19-4a00-ade8-a3ddc6084bcf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:17.705797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e29fc-bc19-4a00-ade8-a3ddc6084bcf None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:17.723220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bf020-fee6-4055-acc7-a3f5dd585253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:17.724743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bf020-fee6-4055-acc7-a3f5dd585253 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:17.725290 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bf020-fee6-4055-acc7-a3f5dd585253 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:17.725770 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bf020-fee6-4055-acc7-a3f5dd585253 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:17.726758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bf020-fee6-4055-acc7-a3f5dd585253 None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:17.727297 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bf020-fee6-4055-acc7-a3f5dd585253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:17.729036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bf020-fee6-4055-acc7-a3f5dd585253 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:17.738023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 44/88] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:29:17 2022] POST /identity/v3/users => generated 289 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:29:17.740609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e29fc-bc19-4a00-ade8-a3ddc6084bcf None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:17.745697 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 45/89] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:17.746792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bf020-fee6-4055-acc7-a3f5dd585253 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:17.789386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:17.820061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:17.935854 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bf020-fee6-4055-acc7-a3f5dd585253 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:17.938958 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e29fc-bc19-4a00-ade8-a3ddc6084bcf None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:17.994562 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e038830-3e6f-466d-8e5c-0917e3d9c52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:17.995216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e038830-3e6f-466d-8e5c-0917e3d9c52a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:17.995493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e038830-3e6f-466d-8e5c-0917e3d9c52a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:17.995791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e038830-3e6f-466d-8e5c-0917e3d9c52a None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:17.997055 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e038830-3e6f-466d-8e5c-0917e3d9c52a None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:17.998612 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e038830-3e6f-466d-8e5c-0917e3d9c52a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:18.000885 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 46/90] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:17 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:18.001690 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e038830-3e6f-466d-8e5c-0917e3d9c52a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:18.011580 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57deec4-8f59-4fbd-8ae3-93bf449fa0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:18.013956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57deec4-8f59-4fbd-8ae3-93bf449fa0eb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:18.014379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57deec4-8f59-4fbd-8ae3-93bf449fa0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:18.014659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57deec4-8f59-4fbd-8ae3-93bf449fa0eb None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:18.016692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57deec4-8f59-4fbd-8ae3-93bf449fa0eb None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:18.019365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57deec4-8f59-4fbd-8ae3-93bf449fa0eb None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:18.022930 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 45/91] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:17 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:18.024367 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57deec4-8f59-4fbd-8ae3-93bf449fa0eb None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:18.178677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:18.183742 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e038830-3e6f-466d-8e5c-0917e3d9c52a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:18.191795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57deec4-8f59-4fbd-8ae3-93bf449fa0eb None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:18.196383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:18.243050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461927a-982e-419a-8388-418b9fcf3088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:18.244090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461927a-982e-419a-8388-418b9fcf3088 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:18.244543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461927a-982e-419a-8388-418b9fcf3088 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:18.244867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461927a-982e-419a-8388-418b9fcf3088 None admin] PATH_INFO: `/v3/users/3829732fccee407fa473d0c55a5aa1d1` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:18.245624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e8662-30ef-4147-9749-923b68ba26fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:18.246137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461927a-982e-419a-8388-418b9fcf3088 None admin] RBAC: Authorizing `identity:get_user(user_id=3829732fccee407fa473d0c55a5aa1d1)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:18.246243 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e8662-30ef-4147-9749-923b68ba26fe None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:18.246536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e8662-30ef-4147-9749-923b68ba26fe None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:18.246817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e8662-30ef-4147-9749-923b68ba26fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:18.247606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461927a-982e-419a-8388-418b9fcf3088 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:18.247709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e8662-30ef-4147-9749-923b68ba26fe None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=998a52ff5527415da541925c79fddaab, user.id=fb1a77890b95445982cf98d4ad77cb92) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:18.247935 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e8662-30ef-4147-9749-923b68ba26fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:18.249403 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e8662-30ef-4147-9749-923b68ba26fe None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:18.249788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 47/92] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:18 2022] GET /identity/v3/users/3829732fccee407fa473d0c55a5aa1d1 => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:18.250717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461927a-982e-419a-8388-418b9fcf3088 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:18.259164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 46/93] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:18 2022] GET /identity/v3/role_assignments?user.id=fb1a77890b95445982cf98d4ad77cb92&scope.project.id=998a52ff5527415da541925c79fddaab&role.id=01714797cab143e1a7a150c068fa9315 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:18.260307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e8662-30ef-4147-9749-923b68ba26fe None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:18.449107 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461927a-982e-419a-8388-418b9fcf3088 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:18.466202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e8662-30ef-4147-9749-923b68ba26fe None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:18.515278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ebf37-80ea-451d-b6f9-13457476c17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:18.515987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ebf37-80ea-451d-b6f9-13457476c17c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:18.516150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ebf37-80ea-451d-b6f9-13457476c17c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:18.516407 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ebf37-80ea-451d-b6f9-13457476c17c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:18.517231 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ebf37-80ea-451d-b6f9-13457476c17c None admin] RBAC: Adding query filter params (role.id=cde4978cbb9648639ff720b5ca82886b, scope.project.id=998a52ff5527415da541925c79fddaab, user.id=3829732fccee407fa473d0c55a5aa1d1) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:18.517594 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ebf37-80ea-451d-b6f9-13457476c17c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:18.519095 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ebf37-80ea-451d-b6f9-13457476c17c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:18.539776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 48/94] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:18 2022] GET /identity/v3/role_assignments?user.id=3829732fccee407fa473d0c55a5aa1d1&scope.project.id=998a52ff5527415da541925c79fddaab&role.id=cde4978cbb9648639ff720b5ca82886b => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:18.541228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484b3b3-8627-43d9-b99d-01a8fd2cb6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:18.542161 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484b3b3-8627-43d9-b99d-01a8fd2cb6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:18.542673 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484b3b3-8627-43d9-b99d-01a8fd2cb6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:18.543161 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484b3b3-8627-43d9-b99d-01a8fd2cb6e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:18.613091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:18.683807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c484b3b3-8627-43d9-b99d-01a8fd2cb6e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:18.690349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c484b3b3-8627-43d9-b99d-01a8fd2cb6e3 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:18.831605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 47/95] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:29:18.837418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ebf37-80ea-451d-b6f9-13457476c17c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:18.883849 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:18.998418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ebf37-80ea-451d-b6f9-13457476c17c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:19.079636 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b18acc-ed85-4281-b902-5afc7fde2d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:19.080792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b18acc-ed85-4281-b902-5afc7fde2d6c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:19.081282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b18acc-ed85-4281-b902-5afc7fde2d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:19.081838 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b18acc-ed85-4281-b902-5afc7fde2d6c None admin] PATH_INFO: `/v3/projects/9fca9db718a1487791d7161b2f653234` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:19.084586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b18acc-ed85-4281-b902-5afc7fde2d6c None admin] RBAC: Authorizing `identity:get_project(project_id=9fca9db718a1487791d7161b2f653234)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:19.087472 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b18acc-ed85-4281-b902-5afc7fde2d6c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:19.091470 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 49/96] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:18 2022] GET /identity/v3/projects/9fca9db718a1487791d7161b2f653234 => generated 312 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:19.100516 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484b3b3-8627-43d9-b99d-01a8fd2cb6e3 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:19.259587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:19.330950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484b3b3-8627-43d9-b99d-01a8fd2cb6e3 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:19.400999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b3e24-fbb2-4059-8aef-41ea578b3858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:19.401964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b3e24-fbb2-4059-8aef-41ea578b3858 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:19.402457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b3e24-fbb2-4059-8aef-41ea578b3858 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:19.402958 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b3e24-fbb2-4059-8aef-41ea578b3858 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:19.404428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b3e24-fbb2-4059-8aef-41ea578b3858 None admin] RBAC: Adding query filter params (role.id=82be5cfedcb3407f8d14522ef87247ac, scope.project.id=9fca9db718a1487791d7161b2f653234, user.id=3829732fccee407fa473d0c55a5aa1d1) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:19.404978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b3e24-fbb2-4059-8aef-41ea578b3858 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:19.407778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b3e24-fbb2-4059-8aef-41ea578b3858 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:19.418902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 48/97] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:19 2022] GET /identity/v3/role_assignments?user.id=3829732fccee407fa473d0c55a5aa1d1&scope.project.id=9fca9db718a1487791d7161b2f653234&role.id=82be5cfedcb3407f8d14522ef87247ac => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:19.749793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48494b37-67f0-49c4-acde-a58130ef8e48 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:19.750303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48494b37-67f0-49c4-acde-a58130ef8e48 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:19.750400 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48494b37-67f0-49c4-acde-a58130ef8e48 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:19.752059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 50/98] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:19.758222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371a893-bb2c-4458-aff5-7652d2def15c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:19.758735 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371a893-bb2c-4458-aff5-7652d2def15c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:19.758832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371a893-bb2c-4458-aff5-7652d2def15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:19.784110 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a371a893-bb2c-4458-aff5-7652d2def15c None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:19.786095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a371a893-bb2c-4458-aff5-7652d2def15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:19.899343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 49/99] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:19.934654 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48494b37-67f0-49c4-acde-a58130ef8e48 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:20.063037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:20.068767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48494b37-67f0-49c4-acde-a58130ef8e48 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:20.116835 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839dc2a-6ca9-4a4d-9a35-a56bf25ffb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:20.117524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839dc2a-6ca9-4a4d-9a35-a56bf25ffb31 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:20.117817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839dc2a-6ca9-4a4d-9a35-a56bf25ffb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:20.118099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839dc2a-6ca9-4a4d-9a35-a56bf25ffb31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:20.133737 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839dc2a-6ca9-4a4d-9a35-a56bf25ffb31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-3839dc2a-6ca9-4a4d-9a35-a56bf25ffb31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:20.139385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:20.140242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:20.141404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:20.141404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:20.141404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:20.141404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:20.141404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:20.141404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:20.141404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:20.141404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:20.141404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:20.141404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:20.141404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:20.141404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:20.141404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 51/100] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:29:20.148775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371a893-bb2c-4458-aff5-7652d2def15c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:20.234039 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:20.286421 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371a893-bb2c-4458-aff5-7652d2def15c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:20.336161 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f16c8-8997-421e-918d-9fc8e22d3384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:20.336815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f16c8-8997-421e-918d-9fc8e22d3384 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:20.337101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f16c8-8997-421e-918d-9fc8e22d3384 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:20.337382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f16c8-8997-421e-918d-9fc8e22d3384 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:20.338437 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f16c8-8997-421e-918d-9fc8e22d3384 None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:20.338755 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f16c8-8997-421e-918d-9fc8e22d3384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:20.340189 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f16c8-8997-421e-918d-9fc8e22d3384 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:20.350135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 50/101] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:20.355651 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839dc2a-6ca9-4a4d-9a35-a56bf25ffb31 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:20.488993 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839dc2a-6ca9-4a4d-9a35-a56bf25ffb31 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:20.542627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c85280-c32f-4e11-b983-5d7ad4b584ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:20.543307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c85280-c32f-4e11-b983-5d7ad4b584ab None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:20.543586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c85280-c32f-4e11-b983-5d7ad4b584ab None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:20.543870 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c85280-c32f-4e11-b983-5d7ad4b584ab None admin] PATH_INFO: `/v3/users/3829732fccee407fa473d0c55a5aa1d1` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:20.545156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c85280-c32f-4e11-b983-5d7ad4b584ab None admin] RBAC: Authorizing `identity:get_user(user_id=3829732fccee407fa473d0c55a5aa1d1)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:20.546736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c85280-c32f-4e11-b983-5d7ad4b584ab None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:20.548437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 52/102] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:20 2022] GET /identity/v3/users/3829732fccee407fa473d0c55a5aa1d1 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:20.557016 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f16c8-8997-421e-918d-9fc8e22d3384 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:20.706942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f16c8-8997-421e-918d-9fc8e22d3384 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:20.746902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:20.762397 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589c19f-446f-403a-a704-c07b45ff0bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:20.763164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589c19f-446f-403a-a704-c07b45ff0bba None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:20.763530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589c19f-446f-403a-a704-c07b45ff0bba None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:20.763823 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589c19f-446f-403a-a704-c07b45ff0bba None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:20.765167 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589c19f-446f-403a-a704-c07b45ff0bba None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:20.766706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589c19f-446f-403a-a704-c07b45ff0bba None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:20.768461 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 51/103] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:20 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:20.777728 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c85280-c32f-4e11-b983-5d7ad4b584ab None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:20.815743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:20.939387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c85280-c32f-4e11-b983-5d7ad4b584ab None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:20.984136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614386c0-1e19-4fbc-a691-3acc7eb0ff67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:20.984845 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614386c0-1e19-4fbc-a691-3acc7eb0ff67 None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:20.985137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614386c0-1e19-4fbc-a691-3acc7eb0ff67 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:20.985423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614386c0-1e19-4fbc-a691-3acc7eb0ff67 None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab/users/3829732fccee407fa473d0c55a5aa1d1/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:20.987537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614386c0-1e19-4fbc-a691-3acc7eb0ff67 None admin] RBAC: Authorizing `identity:create_grant(project_id=998a52ff5527415da541925c79fddaab, user_id=3829732fccee407fa473d0c55a5aa1d1, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:20.989579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614386c0-1e19-4fbc-a691-3acc7eb0ff67 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:21.019770 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 53/104] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:29:20 2022] PUT /identity/v3/projects/998a52ff5527415da541925c79fddaab/users/3829732fccee407fa473d0c55a5aa1d1/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:29:21.400305 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694b345-7ad8-450e-b3fc-b91322c2a022 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:21.401286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694b345-7ad8-450e-b3fc-b91322c2a022 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:21.402116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694b345-7ad8-450e-b3fc-b91322c2a022 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:21.405142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 52/105] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:21.415306 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f41701-1a2e-4b2b-b526-9034a10eacee None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:21.416245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f41701-1a2e-4b2b-b526-9034a10eacee None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:21.417012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f41701-1a2e-4b2b-b526-9034a10eacee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:21.453754 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-11f41701-1a2e-4b2b-b526-9034a10eacee None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:21.456310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-11f41701-1a2e-4b2b-b526-9034a10eacee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:21.611022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 54/106] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:21.707434 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694b345-7ad8-450e-b3fc-b91322c2a022 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:21.972706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694b345-7ad8-450e-b3fc-b91322c2a022 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:22.043696 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86a862-5ce7-4e20-9e5e-45a34c0dfa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:22.044368 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a862-5ce7-4e20-9e5e-45a34c0dfa31 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:22.044653 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a862-5ce7-4e20-9e5e-45a34c0dfa31 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:22.044932 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a862-5ce7-4e20-9e5e-45a34c0dfa31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:22.050333 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86a862-5ce7-4e20-9e5e-45a34c0dfa31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:22.052562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86a862-5ce7-4e20-9e5e-45a34c0dfa31 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:22.054669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 53/107] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:29:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:22.063817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f41701-1a2e-4b2b-b526-9034a10eacee None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:22.106008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:22.267164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f41701-1a2e-4b2b-b526-9034a10eacee None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:22.333881 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c5df7-cfcc-4f56-9bc3-bc41d2da4b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:22.335551 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c5df7-cfcc-4f56-9bc3-bc41d2da4b19 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:22.336365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c5df7-cfcc-4f56-9bc3-bc41d2da4b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:22.337161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c5df7-cfcc-4f56-9bc3-bc41d2da4b19 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:22.339033 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c5df7-cfcc-4f56-9bc3-bc41d2da4b19 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:22.342080 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c5df7-cfcc-4f56-9bc3-bc41d2da4b19 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:22.353293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98d295-cdbb-4f9f-81f3-337c36e14add None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:22.353600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98d295-cdbb-4f9f-81f3-337c36e14add None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:22.353915 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98d295-cdbb-4f9f-81f3-337c36e14add None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:22.355527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 54/108] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:22.362964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860772bf-80f8-43b0-9dbf-5166df9ecdc5 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:22.363365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860772bf-80f8-43b0-9dbf-5166df9ecdc5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:22.363633 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860772bf-80f8-43b0-9dbf-5166df9ecdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:22.402899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-860772bf-80f8-43b0-9dbf-5166df9ecdc5 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:22.404226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-860772bf-80f8-43b0-9dbf-5166df9ecdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:22.433451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 55/109] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 3 10:29:22 2022] POST /identity/v3/projects => generated 302 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:29:22.508979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:22.522792 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 55/110] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:22.586046 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c5df7-cfcc-4f56-9bc3-bc41d2da4b19 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:22.710278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:22.752624 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c5df7-cfcc-4f56-9bc3-bc41d2da4b19 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:22.790338 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b30f8-d971-4b9a-9279-449cc3e5b26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:22.790903 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b30f8-d971-4b9a-9279-449cc3e5b26b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:22.791188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b30f8-d971-4b9a-9279-449cc3e5b26b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:22.791468 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b30f8-d971-4b9a-9279-449cc3e5b26b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:22.803900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b30f8-d971-4b9a-9279-449cc3e5b26b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-619b30f8-d971-4b9a-9279-449cc3e5b26b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:22.809410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:22.811300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:22.813207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:22.813207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:22.813207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:22.813207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:22.813207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:22.813207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:22.813207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:22.813207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:22.813207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:22.813207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:22.813207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:22.813207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:22.813207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 56/111] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:22.821371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860772bf-80f8-43b0-9dbf-5166df9ecdc5 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:22.861981 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786462c0-6945-4189-8990-ee686a8f4276 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:22.862393 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786462c0-6945-4189-8990-ee686a8f4276 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:22.862620 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786462c0-6945-4189-8990-ee686a8f4276 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:22.864804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 57/112] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:29:22.870288 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d0a2f-02b2-494f-9983-020f42e00f01 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:22.870632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d0a2f-02b2-494f-9983-020f42e00f01 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:22.870853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d0a2f-02b2-494f-9983-020f42e00f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:22.891979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ce9d0a2f-02b2-494f-9983-020f42e00f01 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:22.894054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ce9d0a2f-02b2-494f-9983-020f42e00f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:22.965770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860772bf-80f8-43b0-9dbf-5166df9ecdc5 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:23.008187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1dbb03-970e-445e-b85d-f0c8169b1e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:23.009677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dbb03-970e-445e-b85d-f0c8169b1e5a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:23.010186 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dbb03-970e-445e-b85d-f0c8169b1e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:23.010898 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dbb03-970e-445e-b85d-f0c8169b1e5a None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:23.012084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1dbb03-970e-445e-b85d-f0c8169b1e5a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:23.012602 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1dbb03-970e-445e-b85d-f0c8169b1e5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:23.014242 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1dbb03-970e-445e-b85d-f0c8169b1e5a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:23.024930 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 56/113] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:23.037489 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1dbb03-970e-445e-b85d-f0c8169b1e5a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:23.093990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 58/114] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:23.136106 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d0a2f-02b2-494f-9983-020f42e00f01 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:23.210111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:23.303030 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1dbb03-970e-445e-b85d-f0c8169b1e5a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:23.313515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:23.364145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175af6d3-497c-4a3a-b85d-781325be08ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:23.365178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175af6d3-497c-4a3a-b85d-781325be08ac None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:23.365690 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175af6d3-497c-4a3a-b85d-781325be08ac None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:23.366171 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175af6d3-497c-4a3a-b85d-781325be08ac None admin] PATH_INFO: `/v3/users/3d79674fdcd54f0fa0cf65c28d4bbdf8` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:23.396141 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d0a2f-02b2-494f-9983-020f42e00f01 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:23.419980 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175af6d3-497c-4a3a-b85d-781325be08ac None admin] RBAC: Authorizing `identity:get_user(user_id=3d79674fdcd54f0fa0cf65c28d4bbdf8)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:23.422639 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175af6d3-497c-4a3a-b85d-781325be08ac None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:23.425792 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 57/115] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:23 2022] GET /identity/v3/users/3d79674fdcd54f0fa0cf65c28d4bbdf8 => generated 289 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:23.439755 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9407b9-695c-4063-9552-8d1c05476140 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:23.440215 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9407b9-695c-4063-9552-8d1c05476140 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:23.440667 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9407b9-695c-4063-9552-8d1c05476140 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:23.443499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 58/116] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:23 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:23.444875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9407b9-695c-4063-9552-8d1c05476140 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:23.473789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5e662-d328-42f8-870a-faa35713a7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:23.474493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5e662-d328-42f8-870a-faa35713a7af None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:23.474780 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5e662-d328-42f8-870a-faa35713a7af None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:23.475067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5e662-d328-42f8-870a-faa35713a7af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:23.501296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5e662-d328-42f8-870a-faa35713a7af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-7dd5e662-d328-42f8-870a-faa35713a7af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:23.507433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:23.508350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:23.509102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:23.509102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:23.509102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:23.509102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:23.509102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:23.509102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:23.509102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:23.509102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:23.509102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:23.509102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:23.509102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:23.509102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:23.521781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 59/117] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:29:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:29:23.523975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd54353-c1b7-410b-9b59-b9ceacfba506 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:23.527241 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd54353-c1b7-410b-9b59-b9ceacfba506 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:23.529964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd54353-c1b7-410b-9b59-b9ceacfba506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:23.549735 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-bdd54353-c1b7-410b-9b59-b9ceacfba506 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:23.553871 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bdd54353-c1b7-410b-9b59-b9ceacfba506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:23.641129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9407b9-695c-4063-9552-8d1c05476140 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:23.666830 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 60/118] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:23.668447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cdf85-9cb8-4550-96a8-9b4e20a0c080 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:23.674203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cdf85-9cb8-4550-96a8-9b4e20a0c080 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:23.674959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cdf85-9cb8-4550-96a8-9b4e20a0c080 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:23.685507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 61/119] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:23 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:23.685507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cdf85-9cb8-4550-96a8-9b4e20a0c080 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:23.697278 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:23.753007 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02569d29-6cd9-4f0d-814c-acb3b02f33d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:23.754251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02569d29-6cd9-4f0d-814c-acb3b02f33d4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:23.754741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02569d29-6cd9-4f0d-814c-acb3b02f33d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:23.755213 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02569d29-6cd9-4f0d-814c-acb3b02f33d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:23.794857 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:23.864180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02569d29-6cd9-4f0d-814c-acb3b02f33d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:23.867812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02569d29-6cd9-4f0d-814c-acb3b02f33d4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:23.942686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cdf85-9cb8-4550-96a8-9b4e20a0c080 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:23.995203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0a2b0-7b6a-413a-a027-e5b490c60172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:23.996198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0a2b0-7b6a-413a-a027-e5b490c60172 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:23.996706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0a2b0-7b6a-413a-a027-e5b490c60172 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:23.997178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0a2b0-7b6a-413a-a027-e5b490c60172 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:23.998165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0a2b0-7b6a-413a-a027-e5b490c60172 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:23.998694 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0a2b0-7b6a-413a-a027-e5b490c60172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:24.000489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0a2b0-7b6a-413a-a027-e5b490c60172 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:24.012368 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 62/120] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:24.019178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff8b0e-cbc1-4269-b5e4-a0314af30cab None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:24.020270 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff8b0e-cbc1-4269-b5e4-a0314af30cab None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:24.020749 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff8b0e-cbc1-4269-b5e4-a0314af30cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:24.056395 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2aff8b0e-cbc1-4269-b5e4-a0314af30cab None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:24.058441 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 59/121] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:29:24.058926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2aff8b0e-cbc1-4269-b5e4-a0314af30cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:24.060157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02569d29-6cd9-4f0d-814c-acb3b02f33d4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:24.087599 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:24.143962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 63/122] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:24.146003 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff8b0e-cbc1-4269-b5e4-a0314af30cab None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:24.234191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:24.275351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02569d29-6cd9-4f0d-814c-acb3b02f33d4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:24.329770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe28349-1af1-431c-a43d-858aaa4af3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:24.330819 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe28349-1af1-431c-a43d-858aaa4af3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:24.331311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe28349-1af1-431c-a43d-858aaa4af3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:24.331790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe28349-1af1-431c-a43d-858aaa4af3f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:24.345000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe28349-1af1-431c-a43d-858aaa4af3f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-dfe28349-1af1-431c-a43d-858aaa4af3f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:24.350620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:24.351595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:24.352377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:24.352377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:24.352377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:24.352377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:24.352377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:24.352377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:24.352377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:24.352377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:24.352377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:24.352377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:24.352377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:29:24.352377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:24.352967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff8b0e-cbc1-4269-b5e4-a0314af30cab None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:24.355082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 60/123] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:29:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:24.356296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe28349-1af1-431c-a43d-858aaa4af3f9 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:24.427240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28df7f-17e7-4974-ae87-4ca7d8487197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:24.428315 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28df7f-17e7-4974-ae87-4ca7d8487197 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:24.428779 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28df7f-17e7-4974-ae87-4ca7d8487197 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:24.429284 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28df7f-17e7-4974-ae87-4ca7d8487197 None admin] PATH_INFO: `/v3/users/fb1a77890b95445982cf98d4ad77cb92` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:24.431739 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28df7f-17e7-4974-ae87-4ca7d8487197 None admin] RBAC: Authorizing `identity:get_user(user_id=fb1a77890b95445982cf98d4ad77cb92)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:24.434595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28df7f-17e7-4974-ae87-4ca7d8487197 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:24.437999 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 64/124] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:24 2022] GET /identity/v3/users/fb1a77890b95445982cf98d4ad77cb92 => generated 247 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:24.440440 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:24.440930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28df7f-17e7-4974-ae87-4ca7d8487197 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:24.483244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe28349-1af1-431c-a43d-858aaa4af3f9 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:24.523685 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b40563-99eb-4fd2-ac5c-745ea9d64b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:24.524356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b40563-99eb-4fd2-ac5c-745ea9d64b9e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:24.524716 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b40563-99eb-4fd2-ac5c-745ea9d64b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:24.525074 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b40563-99eb-4fd2-ac5c-745ea9d64b9e None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:24.526985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b40563-99eb-4fd2-ac5c-745ea9d64b9e None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:24.528333 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b40563-99eb-4fd2-ac5c-745ea9d64b9e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:24.530719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 61/125] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:24 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:24.531750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b40563-99eb-4fd2-ac5c-745ea9d64b9e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:24.652234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b40563-99eb-4fd2-ac5c-745ea9d64b9e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:24.674220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28df7f-17e7-4974-ae87-4ca7d8487197 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:24.696850 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310eb3b3-ec1c-4a24-9936-94fcac9def25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:24.697644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eb3b3-ec1c-4a24-9936-94fcac9def25 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:24.698007 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eb3b3-ec1c-4a24-9936-94fcac9def25 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:24.698380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eb3b3-ec1c-4a24-9936-94fcac9def25 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:24.699230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310eb3b3-ec1c-4a24-9936-94fcac9def25 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:24.699617 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310eb3b3-ec1c-4a24-9936-94fcac9def25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:24.701190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310eb3b3-ec1c-4a24-9936-94fcac9def25 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:24.708102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:24.708974 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:24.713882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 62/126] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:29:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:24.714765 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310eb3b3-ec1c-4a24-9936-94fcac9def25 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:24.771746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15d51f-3e0f-4398-ae8b-86d757e6218a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:24.773379 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15d51f-3e0f-4398-ae8b-86d757e6218a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:24.774259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15d51f-3e0f-4398-ae8b-86d757e6218a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:24.775170 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15d51f-3e0f-4398-ae8b-86d757e6218a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:24.798834 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15d51f-3e0f-4398-ae8b-86d757e6218a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-6a15d51f-3e0f-4398-ae8b-86d757e6218a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:24.810208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:24.811892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:24.813367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:24.813367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:24.813367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:24.813367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:24.813367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:24.813367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:24.813367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:24.813367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:24.813367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:24.813367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:24.813367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:24.813367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:24.826499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 65/127] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:24.828961 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15d51f-3e0f-4398-ae8b-86d757e6218a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:24.836440 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310eb3b3-ec1c-4a24-9936-94fcac9def25 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:24.903932 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964d0ce-e4bd-4e17-b534-304b8dada111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:24.905521 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964d0ce-e4bd-4e17-b534-304b8dada111 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:24.906285 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964d0ce-e4bd-4e17-b534-304b8dada111 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:24.907017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964d0ce-e4bd-4e17-b534-304b8dada111 None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:24.909721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964d0ce-e4bd-4e17-b534-304b8dada111 None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:24.914133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964d0ce-e4bd-4e17-b534-304b8dada111 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:24.918143 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 63/128] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:24 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:24.920479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964d0ce-e4bd-4e17-b534-304b8dada111 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:24.967427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15d51f-3e0f-4398-ae8b-86d757e6218a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:25.015288 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbb2d8-7e30-495f-8a79-823dda4cebf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:25.016662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbb2d8-7e30-495f-8a79-823dda4cebf2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:25.016783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbb2d8-7e30-495f-8a79-823dda4cebf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:25.017232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbb2d8-7e30-495f-8a79-823dda4cebf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:25.018725 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbb2d8-7e30-495f-8a79-823dda4cebf2 None admin] RBAC: Adding query filter params (role.id=66ef9c3aa98247eab1c1c726f2f6a5a5, scope.project.id=998a52ff5527415da541925c79fddaab, user.id=3d79674fdcd54f0fa0cf65c28d4bbdf8) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:25.019793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbb2d8-7e30-495f-8a79-823dda4cebf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:25.022493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbb2d8-7e30-495f-8a79-823dda4cebf2 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:25.032332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:25.032743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 66/129] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:24 2022] GET /identity/v3/role_assignments?user.id=3d79674fdcd54f0fa0cf65c28d4bbdf8&scope.project.id=998a52ff5527415da541925c79fddaab&role.id=66ef9c3aa98247eab1c1c726f2f6a5a5 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:25.034569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbb2d8-7e30-495f-8a79-823dda4cebf2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:25.093518 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964d0ce-e4bd-4e17-b534-304b8dada111 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:25.108662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:25.147472 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e480a4-a0ab-43a9-a130-46f9ded5d407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:25.148441 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e480a4-a0ab-43a9-a130-46f9ded5d407 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:25.148441 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e480a4-a0ab-43a9-a130-46f9ded5d407 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:25.148729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e480a4-a0ab-43a9-a130-46f9ded5d407 None admin] PATH_INFO: `/v3/users/3829732fccee407fa473d0c55a5aa1d1` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:25.150099 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e480a4-a0ab-43a9-a130-46f9ded5d407 None admin] RBAC: Authorizing `identity:get_user(user_id=3829732fccee407fa473d0c55a5aa1d1)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:25.151630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e480a4-a0ab-43a9-a130-46f9ded5d407 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:25.157554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 64/130] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:24 2022] GET /identity/v3/users/3829732fccee407fa473d0c55a5aa1d1 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:25.157554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e480a4-a0ab-43a9-a130-46f9ded5d407 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:25.250752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbb2d8-7e30-495f-8a79-823dda4cebf2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:25.282899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e480a4-a0ab-43a9-a130-46f9ded5d407 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:25.312422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2be64-fe61-4584-8aba-043676ebc9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:25.313366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2be64-fe61-4584-8aba-043676ebc9bd None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:25.313684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2be64-fe61-4584-8aba-043676ebc9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:25.313969 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2be64-fe61-4584-8aba-043676ebc9bd None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:25.314876 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2be64-fe61-4584-8aba-043676ebc9bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:25.315548 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2be64-fe61-4584-8aba-043676ebc9bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:25.317834 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2be64-fe61-4584-8aba-043676ebc9bd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:25.320633 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae672c-67a6-4709-aa1f-783d3055c236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:25.321333 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae672c-67a6-4709-aa1f-783d3055c236 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:25.321647 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae672c-67a6-4709-aa1f-783d3055c236 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:25.321926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae672c-67a6-4709-aa1f-783d3055c236 None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab/users/fb1a77890b95445982cf98d4ad77cb92/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:25.323966 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae672c-67a6-4709-aa1f-783d3055c236 None admin] RBAC: Authorizing `identity:create_grant(project_id=998a52ff5527415da541925c79fddaab, user_id=fb1a77890b95445982cf98d4ad77cb92, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:25.325489 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae672c-67a6-4709-aa1f-783d3055c236 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:25.329025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 67/131] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:25.331172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db2456-40ec-4ba8-be2b-34afe2f964ca None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:25.331808 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db2456-40ec-4ba8-be2b-34afe2f964ca None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:25.332705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db2456-40ec-4ba8-be2b-34afe2f964ca None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:25.338527 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 68/132] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:25.338527 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6caad95-a738-453a-bcab-1e1040a8a5d5 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:25.338527 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6caad95-a738-453a-bcab-1e1040a8a5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:25.338527 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6caad95-a738-453a-bcab-1e1040a8a5d5 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:25.338527 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 69/133] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:25.340604 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:25.342241 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6caad95-a738-453a-bcab-1e1040a8a5d5 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:25.357555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 65/134] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:29:25 2022] PUT /identity/v3/projects/998a52ff5527415da541925c79fddaab/users/fb1a77890b95445982cf98d4ad77cb92/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:29:25.358716 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae672c-67a6-4709-aa1f-783d3055c236 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:25.571077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:25.586794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6caad95-a738-453a-bcab-1e1040a8a5d5 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:25.611015 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae672c-67a6-4709-aa1f-783d3055c236 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:25.658975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e433162-12b4-4109-8cb5-8806a5cad90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:25.660759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e433162-12b4-4109-8cb5-8806a5cad90d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:25.661363 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e433162-12b4-4109-8cb5-8806a5cad90d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:25.662025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e433162-12b4-4109-8cb5-8806a5cad90d None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:25.664476 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e433162-12b4-4109-8cb5-8806a5cad90d None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:25.667720 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e433162-12b4-4109-8cb5-8806a5cad90d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:25.670031 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c848f-b6f3-4798-8e8d-7f73f82c891c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:25.671395 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 70/135] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:25 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:25.671940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c848f-b6f3-4798-8e8d-7f73f82c891c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:25.672719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c848f-b6f3-4798-8e8d-7f73f82c891c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:25.673507 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c848f-b6f3-4798-8e8d-7f73f82c891c None admin] PATH_INFO: `/v3/users/3829732fccee407fa473d0c55a5aa1d1` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:25.673950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f7085-4b73-41a6-aef4-e77d5ca2a18d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:25.674473 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f7085-4b73-41a6-aef4-e77d5ca2a18d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:25.674987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f7085-4b73-41a6-aef4-e77d5ca2a18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:25.677507 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c848f-b6f3-4798-8e8d-7f73f82c891c None admin] RBAC: Authorizing `identity:get_user(user_id=3829732fccee407fa473d0c55a5aa1d1)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:25.681019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c848f-b6f3-4798-8e8d-7f73f82c891c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:25.684745 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 66/136] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:25 2022] GET /identity/v3/users/3829732fccee407fa473d0c55a5aa1d1 => generated 275 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:25.688852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4148a9b-5804-4ccf-b4ad-1a731a6cc756 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:25.689788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4148a9b-5804-4ccf-b4ad-1a731a6cc756 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:25.690553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4148a9b-5804-4ccf-b4ad-1a731a6cc756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:25.731334 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-082f7085-4b73-41a6-aef4-e77d5ca2a18d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:25.733402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-082f7085-4b73-41a6-aef4-e77d5ca2a18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:25.736209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c4148a9b-5804-4ccf-b4ad-1a731a6cc756 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:25.738894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c4148a9b-5804-4ccf-b4ad-1a731a6cc756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:25.852460 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:25.869732 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 67/137] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:25.871842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4148a9b-5804-4ccf-b4ad-1a731a6cc756 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:25.872987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 71/138] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:25.874301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f7085-4b73-41a6-aef4-e77d5ca2a18d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:26.092919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:26.114956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f7085-4b73-41a6-aef4-e77d5ca2a18d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:26.119367 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4148a9b-5804-4ccf-b4ad-1a731a6cc756 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:26.161670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa39d6-f5c5-4105-ac8c-bfc00b320ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:26.163052 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa39d6-f5c5-4105-ac8c-bfc00b320ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:26.163353 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa39d6-f5c5-4105-ac8c-bfc00b320ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:26.163642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa39d6-f5c5-4105-ac8c-bfc00b320ce3 None admin] PATH_INFO: `/v3/projects/9fca9db718a1487791d7161b2f653234` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:26.164943 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa39d6-f5c5-4105-ac8c-bfc00b320ce3 None admin] RBAC: Authorizing `identity:get_project(project_id=9fca9db718a1487791d7161b2f653234)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:26.166789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa39d6-f5c5-4105-ac8c-bfc00b320ce3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:26.168560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 72/139] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:25 2022] GET /identity/v3/projects/9fca9db718a1487791d7161b2f653234 => generated 312 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:26.170430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa39d6-f5c5-4105-ac8c-bfc00b320ce3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:26.199970 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a195b-68a8-4cb6-ab61-1f4a15963224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:26.201648 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a195b-68a8-4cb6-ab61-1f4a15963224 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:26.202403 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a195b-68a8-4cb6-ab61-1f4a15963224 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:26.203178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a195b-68a8-4cb6-ab61-1f4a15963224 None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab/users/3829732fccee407fa473d0c55a5aa1d1/roles/cde4978cbb9648639ff720b5ca82886b` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:26.208399 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a195b-68a8-4cb6-ab61-1f4a15963224 None admin] RBAC: Authorizing `identity:create_grant(project_id=998a52ff5527415da541925c79fddaab, user_id=3829732fccee407fa473d0c55a5aa1d1, role_id=cde4978cbb9648639ff720b5ca82886b)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:26.211586 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a195b-68a8-4cb6-ab61-1f4a15963224 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:26.236698 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 68/140] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:29:25 2022] PUT /identity/v3/projects/998a52ff5527415da541925c79fddaab/users/3829732fccee407fa473d0c55a5aa1d1/roles/cde4978cbb9648639ff720b5ca82886b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:29:26.239746 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a195b-68a8-4cb6-ab61-1f4a15963224 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:26.292806 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa39d6-f5c5-4105-ac8c-bfc00b320ce3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:26.335869 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e3b4a-f7c7-4a53-a277-ca1017078b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:26.336591 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e3b4a-f7c7-4a53-a277-ca1017078b4e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:26.336886 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e3b4a-f7c7-4a53-a277-ca1017078b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:26.337158 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e3b4a-f7c7-4a53-a277-ca1017078b4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:26.338740 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e3b4a-f7c7-4a53-a277-ca1017078b4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:26.340380 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e3b4a-f7c7-4a53-a277-ca1017078b4e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:26.342138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 73/141] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:29:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:26.342937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e3b4a-f7c7-4a53-a277-ca1017078b4e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:26.370610 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:26.377113 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:26.490336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a195b-68a8-4cb6-ab61-1f4a15963224 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:26.546248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e3b4a-f7c7-4a53-a277-ca1017078b4e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:26.571253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25dea64-5355-4096-b1f9-0e8683c8415b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:26.572823 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25dea64-5355-4096-b1f9-0e8683c8415b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:26.573603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25dea64-5355-4096-b1f9-0e8683c8415b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:26.574482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25dea64-5355-4096-b1f9-0e8683c8415b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:26.590283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d65f11-e04b-4f2e-a8c9-64b8d1459abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:26.591094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d65f11-e04b-4f2e-a8c9-64b8d1459abd None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:26.591381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d65f11-e04b-4f2e-a8c9-64b8d1459abd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:26.591664 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d65f11-e04b-4f2e-a8c9-64b8d1459abd None admin] PATH_INFO: `/v3/projects/9fca9db718a1487791d7161b2f653234/users/3829732fccee407fa473d0c55a5aa1d1/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:26.594000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d65f11-e04b-4f2e-a8c9-64b8d1459abd None admin] RBAC: Authorizing `identity:create_grant(project_id=9fca9db718a1487791d7161b2f653234, user_id=3829732fccee407fa473d0c55a5aa1d1, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:26.595545 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d65f11-e04b-4f2e-a8c9-64b8d1459abd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:26.599615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25dea64-5355-4096-b1f9-0e8683c8415b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-a25dea64-5355-4096-b1f9-0e8683c8415b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:26.611900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:26.619866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:26.622269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:26.622269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:26.622269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:26.622269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:26.622269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:26.622269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:26.622269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:26.622269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:26.622269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:26.622269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:26.622269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:26.622269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:26.624638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 74/142] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:29:26 2022] PUT /identity/v3/projects/9fca9db718a1487791d7161b2f653234/users/3829732fccee407fa473d0c55a5aa1d1/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:29:26.624638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d65f11-e04b-4f2e-a8c9-64b8d1459abd None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:26.625102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 69/143] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:26.637580 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25dea64-5355-4096-b1f9-0e8683c8415b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:26.786594 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:26.799852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d65f11-e04b-4f2e-a8c9-64b8d1459abd None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:26.921681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25dea64-5355-4096-b1f9-0e8683c8415b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:26.957436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:26.974073 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929418ec-9794-4f54-ab30-25d4a1c00fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:26.977010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929418ec-9794-4f54-ab30-25d4a1c00fd7 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:26.977696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929418ec-9794-4f54-ab30-25d4a1c00fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:26.978336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929418ec-9794-4f54-ab30-25d4a1c00fd7 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:26.979806 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929418ec-9794-4f54-ab30-25d4a1c00fd7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:26.982955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929418ec-9794-4f54-ab30-25d4a1c00fd7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:27.034388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa111ae-337a-4f70-ab5e-40720347c345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:27.035630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa111ae-337a-4f70-ab5e-40720347c345 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:27.035926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa111ae-337a-4f70-ab5e-40720347c345 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:27.036201 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa111ae-337a-4f70-ab5e-40720347c345 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:27.036951 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa111ae-337a-4f70-ab5e-40720347c345 None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:27.037267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa111ae-337a-4f70-ab5e-40720347c345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:27.045497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa111ae-337a-4f70-ab5e-40720347c345 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:27.058281 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 75/144] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:29:26 2022] POST /identity/v3/users => generated 283 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:29:27.079643 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 70/145] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:27.088428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929418ec-9794-4f54-ab30-25d4a1c00fd7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:27.489323 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929418ec-9794-4f54-ab30-25d4a1c00fd7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:27.603861 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e74b5-b3c1-4816-adf0-be969497dfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:27.604795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e74b5-b3c1-4816-adf0-be969497dfdb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:27.605255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e74b5-b3c1-4816-adf0-be969497dfdb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:27.605708 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e74b5-b3c1-4816-adf0-be969497dfdb None admin] PATH_INFO: `/v3/users/3829732fccee407fa473d0c55a5aa1d1` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:27.607174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e74b5-b3c1-4816-adf0-be969497dfdb None admin] RBAC: Authorizing `identity:get_user(user_id=3829732fccee407fa473d0c55a5aa1d1)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:27.608812 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e74b5-b3c1-4816-adf0-be969497dfdb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:27.610690 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 76/146] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:27 2022] GET /identity/v3/users/3829732fccee407fa473d0c55a5aa1d1 => generated 275 bytes in 524 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:27.616751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa111ae-337a-4f70-ab5e-40720347c345 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:27.840429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa111ae-337a-4f70-ab5e-40720347c345 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:27.909686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102bf41-8ea0-47a6-9872-1e24c1e0c6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:27.911248 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102bf41-8ea0-47a6-9872-1e24c1e0c6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:27.912081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102bf41-8ea0-47a6-9872-1e24c1e0c6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:27.912829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102bf41-8ea0-47a6-9872-1e24c1e0c6b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:28.003524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:28.058082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102bf41-8ea0-47a6-9872-1e24c1e0c6b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:28.061291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102bf41-8ea0-47a6-9872-1e24c1e0c6b0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:28.270517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 71/147] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 10:29:28.287037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e74b5-b3c1-4816-adf0-be969497dfdb None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:28.337341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:28.470882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e74b5-b3c1-4816-adf0-be969497dfdb None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:28.517268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce83452-1f91-45f3-a3f9-e1fef20dcd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:28.518246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce83452-1f91-45f3-a3f9-e1fef20dcd4a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:28.518246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce83452-1f91-45f3-a3f9-e1fef20dcd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:28.518522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce83452-1f91-45f3-a3f9-e1fef20dcd4a None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:28.520390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce83452-1f91-45f3-a3f9-e1fef20dcd4a None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:28.521978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce83452-1f91-45f3-a3f9-e1fef20dcd4a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:28.523983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 77/148] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:28 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:28.533445 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102bf41-8ea0-47a6-9872-1e24c1e0c6b0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:28.753336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:28.803435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102bf41-8ea0-47a6-9872-1e24c1e0c6b0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:28.873403 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eeb81a-d7c2-40a7-9d92-34a9c7a714ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:28.875393 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eeb81a-d7c2-40a7-9d92-34a9c7a714ad None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:28.876261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eeb81a-d7c2-40a7-9d92-34a9c7a714ad None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:28.877062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eeb81a-d7c2-40a7-9d92-34a9c7a714ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:28.879523 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eeb81a-d7c2-40a7-9d92-34a9c7a714ad None admin] RBAC: Adding query filter params (role.id=82be5cfedcb3407f8d14522ef87247ac, scope.project.id=998a52ff5527415da541925c79fddaab, user.id=3829732fccee407fa473d0c55a5aa1d1) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:28.880392 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eeb81a-d7c2-40a7-9d92-34a9c7a714ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:28.883545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eeb81a-d7c2-40a7-9d92-34a9c7a714ad None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:28.896421 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 72/149] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:28 2022] GET /identity/v3/role_assignments?user.id=3829732fccee407fa473d0c55a5aa1d1&scope.project.id=998a52ff5527415da541925c79fddaab&role.id=82be5cfedcb3407f8d14522ef87247ac => generated 608 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:29.001958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d51c16-2225-402b-ae3a-8c1da22a42a7 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:29.002374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d51c16-2225-402b-ae3a-8c1da22a42a7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:29.002519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d51c16-2225-402b-ae3a-8c1da22a42a7 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:29.004476 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 78/150] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:29.011743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c480d45-d170-4dc4-8349-ff55d68648b5 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:29.012638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c480d45-d170-4dc4-8349-ff55d68648b5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:29.013407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c480d45-d170-4dc4-8349-ff55d68648b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:29.054254 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8c480d45-d170-4dc4-8349-ff55d68648b5 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:29.057098 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8c480d45-d170-4dc4-8349-ff55d68648b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:29.194026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 73/151] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:29.224777 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d51c16-2225-402b-ae3a-8c1da22a42a7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:29.292013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1194ec-402c-4f9f-b4f4-16c0ce661928 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:29.292639 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1194ec-402c-4f9f-b4f4-16c0ce661928 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:29.293328 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1194ec-402c-4f9f-b4f4-16c0ce661928 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:29.295136 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 74/152] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:29:29.314167 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7df40-4c89-4aa4-ad2b-2ab538e4d5e8 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:29.314443 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7df40-4c89-4aa4-ad2b-2ab538e4d5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:29.314635 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7df40-4c89-4aa4-ad2b-2ab538e4d5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:29.356749 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-89d7df40-4c89-4aa4-ad2b-2ab538e4d5e8 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:29.358008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-89d7df40-4c89-4aa4-ad2b-2ab538e4d5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:29.384863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:29.430505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d51c16-2225-402b-ae3a-8c1da22a42a7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:29.490584 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:29.498031 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 75/153] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:29.514260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1bd66-8706-44a5-9d71-507299a81892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:29.514942 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1bd66-8706-44a5-9d71-507299a81892 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:29.515223 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1bd66-8706-44a5-9d71-507299a81892 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:29.515499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1bd66-8706-44a5-9d71-507299a81892 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:29.531327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f1bd66-8706-44a5-9d71-507299a81892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-d7f1bd66-8706-44a5-9d71-507299a81892 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:29.537678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:29.538553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:29.539756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:29.539756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:29.539756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:29.539756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:29.539756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:29.539756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:29.539756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:29.539756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:29.539756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:29.539756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:29.539756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:29.539756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:29.539756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 79/154] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:29:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:29.544803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7df40-4c89-4aa4-ad2b-2ab538e4d5e8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:29.595633 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d22e37-808e-4407-a370-b48de95da848 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:29.595633 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d22e37-808e-4407-a370-b48de95da848 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:29.596094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d22e37-808e-4407-a370-b48de95da848 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:29.597331 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 80/155] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:29:29.600920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d22e37-808e-4407-a370-b48de95da848 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:29.695815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7df40-4c89-4aa4-ad2b-2ab538e4d5e8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:29.767644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667a3fa-dbdf-49c9-891b-e917ec19a65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:29.768728 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667a3fa-dbdf-49c9-891b-e917ec19a65a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:29.769304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667a3fa-dbdf-49c9-891b-e917ec19a65a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:29.770281 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667a3fa-dbdf-49c9-891b-e917ec19a65a None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:29.771519 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667a3fa-dbdf-49c9-891b-e917ec19a65a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:29.772051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667a3fa-dbdf-49c9-891b-e917ec19a65a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:29.774654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667a3fa-dbdf-49c9-891b-e917ec19a65a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:29.794319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d22e37-808e-4407-a370-b48de95da848 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:29.797688 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 76/156] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:29.798814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f876d8f-6c64-4718-a367-e37cf671f02a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:29.799178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f876d8f-6c64-4718-a367-e37cf671f02a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:29.800524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f876d8f-6c64-4718-a367-e37cf671f02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:29.837151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-9f876d8f-6c64-4718-a367-e37cf671f02a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:29.838892 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9f876d8f-6c64-4718-a367-e37cf671f02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:29.864244 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb919bcc-7de5-4a2a-b9dd-e90185977858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:29.865496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb919bcc-7de5-4a2a-b9dd-e90185977858 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:29.866889 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb919bcc-7de5-4a2a-b9dd-e90185977858 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:29.867562 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb919bcc-7de5-4a2a-b9dd-e90185977858 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:29.887479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:29.897361 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb919bcc-7de5-4a2a-b9dd-e90185977858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-cb919bcc-7de5-4a2a-b9dd-e90185977858 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:29.904492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:29.905535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:29.906336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:29.906336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:29.906336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:29.906336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:29.906336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:29.906336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:29.906336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:29.906336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:29.906336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:29.906336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:29.906336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:29.906336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:29.914645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 81/157] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:29:29.915368 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb919bcc-7de5-4a2a-b9dd-e90185977858 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:29.959467 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:29.980775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 77/158] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:29.983101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f876d8f-6c64-4718-a367-e37cf671f02a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:30.125316 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb919bcc-7de5-4a2a-b9dd-e90185977858 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:30.186023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f876d8f-6c64-4718-a367-e37cf671f02a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:30.195560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a22a67-e95f-491f-96e3-0e52cfc379d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:30.201435 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a22a67-e95f-491f-96e3-0e52cfc379d2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:30.202019 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a22a67-e95f-491f-96e3-0e52cfc379d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:30.202493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a22a67-e95f-491f-96e3-0e52cfc379d2 None admin] PATH_INFO: `/v3/users/fb1a77890b95445982cf98d4ad77cb92` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:30.204199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a22a67-e95f-491f-96e3-0e52cfc379d2 None admin] RBAC: Authorizing `identity:get_user(user_id=fb1a77890b95445982cf98d4ad77cb92)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:30.206009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a22a67-e95f-491f-96e3-0e52cfc379d2 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:30.208208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 82/159] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:29 2022] GET /identity/v3/users/fb1a77890b95445982cf98d4ad77cb92 => generated 247 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:30.211415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a22a67-e95f-491f-96e3-0e52cfc379d2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:30.262534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc204222-ac7b-41b7-9802-12e61306d5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:30.264740 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc204222-ac7b-41b7-9802-12e61306d5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:30.265526 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc204222-ac7b-41b7-9802-12e61306d5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:30.266615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc204222-ac7b-41b7-9802-12e61306d5e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:30.267637 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc204222-ac7b-41b7-9802-12e61306d5e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:30.268324 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc204222-ac7b-41b7-9802-12e61306d5e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:30.270866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc204222-ac7b-41b7-9802-12e61306d5e2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:30.293621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 78/160] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:30.295034 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc204222-ac7b-41b7-9802-12e61306d5e2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:30.355329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:30.435137 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:30.475913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a22a67-e95f-491f-96e3-0e52cfc379d2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:30.492041 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc204222-ac7b-41b7-9802-12e61306d5e2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:30.541085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5d820-1156-4e80-9138-640a595bd280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:30.542180 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e13c9b-9bc5-4ced-b822-938fe348a8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:30.543423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5d820-1156-4e80-9138-640a595bd280 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:30.544291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e13c9b-9bc5-4ced-b822-938fe348a8ce None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:30.544780 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5d820-1156-4e80-9138-640a595bd280 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:30.545252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5d820-1156-4e80-9138-640a595bd280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:30.549978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e13c9b-9bc5-4ced-b822-938fe348a8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:30.551931 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e13c9b-9bc5-4ced-b822-938fe348a8ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:30.576841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e13c9b-9bc5-4ced-b822-938fe348a8ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-25e13c9b-9bc5-4ced-b822-938fe348a8ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:30.585534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:30.586524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:30.587300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:30.587300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:30.587300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:30.587300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:30.587300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:30.587300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:30.587300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:30.587300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:30.587300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:30.587300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:30.587300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:29:30.587300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:30.587300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 83/161] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:29:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:30.587940 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e13c9b-9bc5-4ced-b822-938fe348a8ce None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:30.663755 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5d820-1156-4e80-9138-640a595bd280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:30.665672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5d820-1156-4e80-9138-640a595bd280 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:30.731906 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:30.786342 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 79/162] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:29:30.793307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd02cc-30eb-4340-b15c-03ec41f12d78 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:30.794335 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd02cc-30eb-4340-b15c-03ec41f12d78 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:30.794819 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd02cc-30eb-4340-b15c-03ec41f12d78 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:30.796493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 80/163] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:30.797637 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd02cc-30eb-4340-b15c-03ec41f12d78 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:30.880959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e13c9b-9bc5-4ced-b822-938fe348a8ce None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:30.936408 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd02cc-30eb-4340-b15c-03ec41f12d78 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:30.957893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:30.980361 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487134a9-dfd5-47f1-b718-38fce8485688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:30.981034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487134a9-dfd5-47f1-b718-38fce8485688 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:30.981314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487134a9-dfd5-47f1-b718-38fce8485688 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:30.981619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487134a9-dfd5-47f1-b718-38fce8485688 None admin] PATH_INFO: `/v3/users/3d79674fdcd54f0fa0cf65c28d4bbdf8` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:30.987986 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487134a9-dfd5-47f1-b718-38fce8485688 None admin] RBAC: Authorizing `identity:get_user(user_id=3d79674fdcd54f0fa0cf65c28d4bbdf8)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:30.989380 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487134a9-dfd5-47f1-b718-38fce8485688 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:30.996184 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 84/164] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:30 2022] GET /identity/v3/users/3d79674fdcd54f0fa0cf65c28d4bbdf8 => generated 289 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:30.997148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487134a9-dfd5-47f1-b718-38fce8485688 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:31.025264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b89b24-4e19-4358-80cc-006134d9a906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:31.026282 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b89b24-4e19-4358-80cc-006134d9a906 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:31.026771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b89b24-4e19-4358-80cc-006134d9a906 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:31.027270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b89b24-4e19-4358-80cc-006134d9a906 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:31.028510 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b89b24-4e19-4358-80cc-006134d9a906 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:31.029045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b89b24-4e19-4358-80cc-006134d9a906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:31.030897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b89b24-4e19-4358-80cc-006134d9a906 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:31.041505 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 81/165] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:29:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:31.043142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca4c83-6edc-4ea6-855d-f72a53b1af99 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:31.043642 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca4c83-6edc-4ea6-855d-f72a53b1af99 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:31.044122 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca4c83-6edc-4ea6-855d-f72a53b1af99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:31.063678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b3ca4c83-6edc-4ea6-855d-f72a53b1af99 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:31.065240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b3ca4c83-6edc-4ea6-855d-f72a53b1af99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:31.082443 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:31.142519 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 82/166] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:31.143684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca4c83-6edc-4ea6-855d-f72a53b1af99 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:31.246853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487134a9-dfd5-47f1-b718-38fce8485688 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:31.297123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca4c83-6edc-4ea6-855d-f72a53b1af99 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:31.340631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde805a-cd91-4ee1-b2c2-bd80b36a9626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:31.341487 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde805a-cd91-4ee1-b2c2-bd80b36a9626 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:31.341882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde805a-cd91-4ee1-b2c2-bd80b36a9626 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:31.342289 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde805a-cd91-4ee1-b2c2-bd80b36a9626 None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:31.343673 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde805a-cd91-4ee1-b2c2-bd80b36a9626 None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:31.345302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde805a-cd91-4ee1-b2c2-bd80b36a9626 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:31.347305 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 83/167] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:31 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:31.348227 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde805a-cd91-4ee1-b2c2-bd80b36a9626 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:31.366798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e16e2-3a3f-436c-b7f3-6d3dc7f07f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:31.367524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e16e2-3a3f-436c-b7f3-6d3dc7f07f1a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:31.367814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e16e2-3a3f-436c-b7f3-6d3dc7f07f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:31.367969 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:31.368152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e16e2-3a3f-436c-b7f3-6d3dc7f07f1a None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:31.369748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e16e2-3a3f-436c-b7f3-6d3dc7f07f1a None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:31.371297 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e16e2-3a3f-436c-b7f3-6d3dc7f07f1a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:31.373599 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 85/168] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:30 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:31.374529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e16e2-3a3f-436c-b7f3-6d3dc7f07f1a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:31.478285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:31.480284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde805a-cd91-4ee1-b2c2-bd80b36a9626 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:31.520371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c6702-2aa5-4885-adf4-5045af4f4613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:31.520953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c6702-2aa5-4885-adf4-5045af4f4613 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:31.521239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c6702-2aa5-4885-adf4-5045af4f4613 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:31.521560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c6702-2aa5-4885-adf4-5045af4f4613 None admin] PATH_INFO: `/v3/users/3829732fccee407fa473d0c55a5aa1d1` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:31.522824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c6702-2aa5-4885-adf4-5045af4f4613 None admin] RBAC: Authorizing `identity:get_user(user_id=3829732fccee407fa473d0c55a5aa1d1)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:31.524295 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c6702-2aa5-4885-adf4-5045af4f4613 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:31.526483 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 84/169] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:31 2022] GET /identity/v3/users/3829732fccee407fa473d0c55a5aa1d1 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:31.527452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c6702-2aa5-4885-adf4-5045af4f4613 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:31.561890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e16e2-3a3f-436c-b7f3-6d3dc7f07f1a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:31.602496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b9de9-a20f-4f12-bbf7-faab593e63bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:31.604051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b9de9-a20f-4f12-bbf7-faab593e63bc None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:31.604824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b9de9-a20f-4f12-bbf7-faab593e63bc None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:31.605313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b9de9-a20f-4f12-bbf7-faab593e63bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:31.620773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b9de9-a20f-4f12-bbf7-faab593e63bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-f20b9de9-a20f-4f12-bbf7-faab593e63bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:31.628234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:31.629426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:31.630508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:31.630508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:31.630508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:31.630508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:31.630508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:31.630508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:31.630508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:31.630508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:31.630508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:31.630508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:31.630508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:31.630508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:31.633247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 86/170] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:31.633435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:31.634993 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b9de9-a20f-4f12-bbf7-faab593e63bc None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:31.657592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c6702-2aa5-4885-adf4-5045af4f4613 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:31.721594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec91abb-6810-4fcf-abc3-bf6546b044d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:31.723135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec91abb-6810-4fcf-abc3-bf6546b044d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:31.723679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec91abb-6810-4fcf-abc3-bf6546b044d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:31.724101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec91abb-6810-4fcf-abc3-bf6546b044d4 None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab/users/3d79674fdcd54f0fa0cf65c28d4bbdf8/roles/66ef9c3aa98247eab1c1c726f2f6a5a5` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:31.726622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec91abb-6810-4fcf-abc3-bf6546b044d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=998a52ff5527415da541925c79fddaab, user_id=3d79674fdcd54f0fa0cf65c28d4bbdf8, role_id=66ef9c3aa98247eab1c1c726f2f6a5a5)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:31.728291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec91abb-6810-4fcf-abc3-bf6546b044d4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:31.747275 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 85/171] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:29:31 2022] PUT /identity/v3/projects/998a52ff5527415da541925c79fddaab/users/3d79674fdcd54f0fa0cf65c28d4bbdf8/roles/66ef9c3aa98247eab1c1c726f2f6a5a5 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:29:31.748615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec91abb-6810-4fcf-abc3-bf6546b044d4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:31.891723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b9de9-a20f-4f12-bbf7-faab593e63bc None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:31.893914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec91abb-6810-4fcf-abc3-bf6546b044d4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:31.940760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ecac0-9e5f-41b2-b55b-a865ede74dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:31.942270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ecac0-9e5f-41b2-b55b-a865ede74dce None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:31.942697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ecac0-9e5f-41b2-b55b-a865ede74dce None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:31.943113 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ecac0-9e5f-41b2-b55b-a865ede74dce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:31.967072 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:31.968126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:32.014113 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369e2f5-6f29-47a3-a692-f1712a2e56b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:32.015063 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369e2f5-6f29-47a3-a692-f1712a2e56b3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:32.015528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369e2f5-6f29-47a3-a692-f1712a2e56b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:32.016470 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369e2f5-6f29-47a3-a692-f1712a2e56b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:32.017916 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369e2f5-6f29-47a3-a692-f1712a2e56b3 None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=998a52ff5527415da541925c79fddaab, user.id=fb1a77890b95445982cf98d4ad77cb92) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:32.018432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369e2f5-6f29-47a3-a692-f1712a2e56b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:32.020996 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369e2f5-6f29-47a3-a692-f1712a2e56b3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:32.029585 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 87/172] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:31 2022] GET /identity/v3/role_assignments?user.id=fb1a77890b95445982cf98d4ad77cb92&scope.project.id=998a52ff5527415da541925c79fddaab&role.id=01714797cab143e1a7a150c068fa9315 => generated 608 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:32.031574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369e2f5-6f29-47a3-a692-f1712a2e56b3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:32.081066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ecac0-9e5f-41b2-b55b-a865ede74dce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:32.083239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ecac0-9e5f-41b2-b55b-a865ede74dce None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:32.182765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369e2f5-6f29-47a3-a692-f1712a2e56b3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:32.230134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 86/173] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:29:32.231767 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3837ab-e46f-48ce-b4df-b33109189dfd None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:32.232289 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3837ab-e46f-48ce-b4df-b33109189dfd None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:32.232770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3837ab-e46f-48ce-b4df-b33109189dfd None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:32.234384 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 87/174] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:32.235850 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccf82a-a71e-45d8-8895-619c9e05d38a None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:32.236363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccf82a-a71e-45d8-8895-619c9e05d38a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:32.236859 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccf82a-a71e-45d8-8895-619c9e05d38a None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:32.238954 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 88/175] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:32.242826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaf13e-37c1-4680-9b33-5685ad54a6b6 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:32.244458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaf13e-37c1-4680-9b33-5685ad54a6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:32.245062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaf13e-37c1-4680-9b33-5685ad54a6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:32.249458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa159e-50c3-4fa0-a7a1-5257f939d1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:32.250457 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa159e-50c3-4fa0-a7a1-5257f939d1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:32.250937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa159e-50c3-4fa0-a7a1-5257f939d1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:32.251405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa159e-50c3-4fa0-a7a1-5257f939d1a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:32.252342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa159e-50c3-4fa0-a7a1-5257f939d1a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:32.252861 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa159e-50c3-4fa0-a7a1-5257f939d1a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:32.255124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa159e-50c3-4fa0-a7a1-5257f939d1a3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:32.265763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 88/176] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:32.267349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba140b9-0095-4b51-88aa-71b637587e9f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:32.267835 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba140b9-0095-4b51-88aa-71b637587e9f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:32.268303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba140b9-0095-4b51-88aa-71b637587e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:32.289309 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0ecaf13e-37c1-4680-9b33-5685ad54a6b6 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:32.291409 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0ecaf13e-37c1-4680-9b33-5685ad54a6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:32.295396 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-eba140b9-0095-4b51-88aa-71b637587e9f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:32.298187 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-eba140b9-0095-4b51-88aa-71b637587e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:32.367530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:32.374296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:32.413888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 89/177] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:32.415642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba140b9-0095-4b51-88aa-71b637587e9f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:32.438461 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 89/178] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:32.439368 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaf13e-37c1-4680-9b33-5685ad54a6b6 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:32.582265 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba140b9-0095-4b51-88aa-71b637587e9f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:32.623984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaf13e-37c1-4680-9b33-5685ad54a6b6 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:32.642010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6356e4-0a2d-4486-8113-1b884f5f3b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:32.643054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6356e4-0a2d-4486-8113-1b884f5f3b8b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:32.643491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6356e4-0a2d-4486-8113-1b884f5f3b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:32.643913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6356e4-0a2d-4486-8113-1b884f5f3b8b None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:32.647916 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6356e4-0a2d-4486-8113-1b884f5f3b8b None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:32.649439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6356e4-0a2d-4486-8113-1b884f5f3b8b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:32.651294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 90/179] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:32 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:32.652486 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da955ca-6c4b-4949-b058-1053f67be934 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:32.652760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da955ca-6c4b-4949-b058-1053f67be934 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:32.653036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da955ca-6c4b-4949-b058-1053f67be934 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:32.661514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 91/180] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:32.661514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da955ca-6c4b-4949-b058-1053f67be934 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:32.688736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05b207-861a-49a7-861a-e7f166c2422b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:32.688736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05b207-861a-49a7-861a-e7f166c2422b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:32.688736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05b207-861a-49a7-861a-e7f166c2422b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:32.689160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05b207-861a-49a7-861a-e7f166c2422b None admin] PATH_INFO: `/v3/users/3829732fccee407fa473d0c55a5aa1d1` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:32.690516 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05b207-861a-49a7-861a-e7f166c2422b None admin] RBAC: Authorizing `identity:get_user(user_id=3829732fccee407fa473d0c55a5aa1d1)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:32.692212 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05b207-861a-49a7-861a-e7f166c2422b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:32.694302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 90/181] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:32 2022] GET /identity/v3/users/3829732fccee407fa473d0c55a5aa1d1 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:32.694765 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05b207-861a-49a7-861a-e7f166c2422b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:32.747344 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:32.749867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:32.830187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05b207-861a-49a7-861a-e7f166c2422b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:32.845449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da955ca-6c4b-4949-b058-1053f67be934 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:32.888654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417efdb-940b-4707-b40d-b30eb342090d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:32.889876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417efdb-940b-4707-b40d-b30eb342090d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:32.890280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417efdb-940b-4707-b40d-b30eb342090d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:32.890745 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417efdb-940b-4707-b40d-b30eb342090d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:32.906930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b186c-40a3-4c15-a456-8541500060e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:32.907962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b186c-40a3-4c15-a456-8541500060e7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:32.908421 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b186c-40a3-4c15-a456-8541500060e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:32.908863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b186c-40a3-4c15-a456-8541500060e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:32.911250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417efdb-940b-4707-b40d-b30eb342090d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:32.911962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b186c-40a3-4c15-a456-8541500060e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:32.914061 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b186c-40a3-4c15-a456-8541500060e7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:32.917029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 92/182] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:29:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:32.920124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b186c-40a3-4c15-a456-8541500060e7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-8417efdb-940b-4707-b40d-b30eb342090d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:32.922053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:32.923611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:32.925032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:32.925032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:32.925032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:32.925032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:32.925032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:32.925032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:32.925032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:32.925032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:32.925032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:32.925032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:32.925032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:32.925032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:32.925032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 91/183] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:29:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:32.927805 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3b718-a5cb-4ae8-b165-925b4c566a8b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:32.928303 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3b718-a5cb-4ae8-b165-925b4c566a8b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:32.928789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3b718-a5cb-4ae8-b165-925b4c566a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:32.956043 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a9a3b718-a5cb-4ae8-b165-925b4c566a8b None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:32.957951 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a9a3b718-a5cb-4ae8-b165-925b4c566a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:33.069548 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 92/184] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:33.069548 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3b718-a5cb-4ae8-b165-925b4c566a8b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:33.098206 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b186c-40a3-4c15-a456-8541500060e7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:33.119840 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:33.134818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:33.171688 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c3ab2-3268-4515-9432-6eadef962e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:33.173065 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c3ab2-3268-4515-9432-6eadef962e14 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:33.173317 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c3ab2-3268-4515-9432-6eadef962e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:33.173908 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c3ab2-3268-4515-9432-6eadef962e14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:33.175529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c3ab2-3268-4515-9432-6eadef962e14 None admin] RBAC: Adding query filter params (role.id=cde4978cbb9648639ff720b5ca82886b, scope.project.id=998a52ff5527415da541925c79fddaab, user.id=3829732fccee407fa473d0c55a5aa1d1) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:33.176212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c3ab2-3268-4515-9432-6eadef962e14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:33.178946 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c3ab2-3268-4515-9432-6eadef962e14 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:33.190708 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 93/185] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:32 2022] GET /identity/v3/role_assignments?user.id=3829732fccee407fa473d0c55a5aa1d1&scope.project.id=998a52ff5527415da541925c79fddaab&role.id=cde4978cbb9648639ff720b5ca82886b => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:33.198595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c3ab2-3268-4515-9432-6eadef962e14 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:33.215471 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3b718-a5cb-4ae8-b165-925b4c566a8b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:33.291842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263418d1-1532-4042-ac2a-28351bb74c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:33.293523 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263418d1-1532-4042-ac2a-28351bb74c5d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:33.294252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263418d1-1532-4042-ac2a-28351bb74c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:33.294972 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263418d1-1532-4042-ac2a-28351bb74c5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:33.340184 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c3ab2-3268-4515-9432-6eadef962e14 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:33.382811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca49fb0-dbcf-4fa3-a184-d8fff5d16caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:33.384427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca49fb0-dbcf-4fa3-a184-d8fff5d16caa None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:33.385146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca49fb0-dbcf-4fa3-a184-d8fff5d16caa None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:33.386892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca49fb0-dbcf-4fa3-a184-d8fff5d16caa None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:33.388649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca49fb0-dbcf-4fa3-a184-d8fff5d16caa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:33.391208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca49fb0-dbcf-4fa3-a184-d8fff5d16caa None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:33.419906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 94/186] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:29:33 2022] POST /identity/v3/users => generated 297 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:29:33.421058 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca49fb0-dbcf-4fa3-a184-d8fff5d16caa None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:33.453085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263418d1-1532-4042-ac2a-28351bb74c5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:33.457308 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263418d1-1532-4042-ac2a-28351bb74c5d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:33.474472 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:33.534573 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:33.595668 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca49fb0-dbcf-4fa3-a184-d8fff5d16caa None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:33.679221 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c0057-8eff-45b8-8626-30e2404a4f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:33.681117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c0057-8eff-45b8-8626-30e2404a4f39 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:33.681950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c0057-8eff-45b8-8626-30e2404a4f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:33.682105 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 93/187] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:29:33.682877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263418d1-1532-4042-ac2a-28351bb74c5d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:33.683291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c0057-8eff-45b8-8626-30e2404a4f39 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:33.684992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c0057-8eff-45b8-8626-30e2404a4f39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:33.690361 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c0057-8eff-45b8-8626-30e2404a4f39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:33.693175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c0057-8eff-45b8-8626-30e2404a4f39 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:33.709234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 95/188] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:33.711986 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c0057-8eff-45b8-8626-30e2404a4f39 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:33.879380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263418d1-1532-4042-ac2a-28351bb74c5d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:33.893550 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:33.900332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c0057-8eff-45b8-8626-30e2404a4f39 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:33.936051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:33.953162 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e97bb-93d1-4ec3-9720-99c8a3beb11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:33.954940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e97bb-93d1-4ec3-9720-99c8a3beb11f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:33.955990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e97bb-93d1-4ec3-9720-99c8a3beb11f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:33.956958 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e97bb-93d1-4ec3-9720-99c8a3beb11f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:33.958954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2564cf-0f9c-4237-bcd9-8b0bfe56121c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:33.959705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2564cf-0f9c-4237-bcd9-8b0bfe56121c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:33.960054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2564cf-0f9c-4237-bcd9-8b0bfe56121c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:33.960399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2564cf-0f9c-4237-bcd9-8b0bfe56121c None admin] PATH_INFO: `/v3/projects/9fca9db718a1487791d7161b2f653234` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:33.963816 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2564cf-0f9c-4237-bcd9-8b0bfe56121c None admin] RBAC: Authorizing `identity:get_project(project_id=9fca9db718a1487791d7161b2f653234)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:33.965702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2564cf-0f9c-4237-bcd9-8b0bfe56121c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:33.972438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 96/189] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:33 2022] GET /identity/v3/projects/9fca9db718a1487791d7161b2f653234 => generated 312 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:33.973580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2564cf-0f9c-4237-bcd9-8b0bfe56121c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:33.982132 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e97bb-93d1-4ec3-9720-99c8a3beb11f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-127e97bb-93d1-4ec3-9720-99c8a3beb11f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:33.993430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:33.995187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:33.996692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:33.996692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:33.996692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:33.996692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:33.996692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:33.996692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:33.996692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:33.996692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:33.996692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:33.996692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:33.996692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:29:33.996692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:34.001392 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 94/190] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:29:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:34.003103 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e97bb-93d1-4ec3-9720-99c8a3beb11f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:34.104468 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2564cf-0f9c-4237-bcd9-8b0bfe56121c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:34.145189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5762bb7-a388-4b97-a8bd-fe723474531d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:34.145787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5762bb7-a388-4b97-a8bd-fe723474531d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:34.146075 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5762bb7-a388-4b97-a8bd-fe723474531d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:34.146352 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5762bb7-a388-4b97-a8bd-fe723474531d None admin] PATH_INFO: `/v3/users/4abb99ab49734e918d15103010c2b0c4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:34.162858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5762bb7-a388-4b97-a8bd-fe723474531d None admin] RBAC: Authorizing `identity:get_user(user_id=4abb99ab49734e918d15103010c2b0c4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:34.164345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5762bb7-a388-4b97-a8bd-fe723474531d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:34.166281 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 97/191] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:33 2022] GET /identity/v3/users/4abb99ab49734e918d15103010c2b0c4 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:34.167188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5762bb7-a388-4b97-a8bd-fe723474531d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:34.198023 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:34.204247 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e97bb-93d1-4ec3-9720-99c8a3beb11f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:34.271433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c3b21-b84e-4598-9966-f53144f49143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:34.272986 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c3b21-b84e-4598-9966-f53144f49143 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:34.273776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c3b21-b84e-4598-9966-f53144f49143 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:34.274556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c3b21-b84e-4598-9966-f53144f49143 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:34.276443 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c3b21-b84e-4598-9966-f53144f49143 None admin] RBAC: Adding query filter params (role.id=82be5cfedcb3407f8d14522ef87247ac, scope.project.id=9fca9db718a1487791d7161b2f653234, user.id=3829732fccee407fa473d0c55a5aa1d1) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:34.277293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c3b21-b84e-4598-9966-f53144f49143 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:34.280388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c3b21-b84e-4598-9966-f53144f49143 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:34.286173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5762bb7-a388-4b97-a8bd-fe723474531d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:34.294003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 95/192] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:34 2022] GET /identity/v3/role_assignments?user.id=3829732fccee407fa473d0c55a5aa1d1&scope.project.id=9fca9db718a1487791d7161b2f653234&role.id=82be5cfedcb3407f8d14522ef87247ac => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:34.295897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c3b21-b84e-4598-9966-f53144f49143 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:34.349084 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564563a7-e736-4baa-a5b3-59e22fa1c725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:34.349707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564563a7-e736-4baa-a5b3-59e22fa1c725 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:34.349990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564563a7-e736-4baa-a5b3-59e22fa1c725 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:34.350272 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564563a7-e736-4baa-a5b3-59e22fa1c725 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:34.351012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564563a7-e736-4baa-a5b3-59e22fa1c725 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:34.351315 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564563a7-e736-4baa-a5b3-59e22fa1c725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:34.352724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564563a7-e736-4baa-a5b3-59e22fa1c725 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:34.367974 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 98/193] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:29:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:34.373787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564563a7-e736-4baa-a5b3-59e22fa1c725 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:34.411615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:34.485452 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:34.495662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c3b21-b84e-4598-9966-f53144f49143 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:34.505628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564563a7-e736-4baa-a5b3-59e22fa1c725 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:34.546833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c66051-97df-4315-92f7-4929447b274c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:34.547911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c66051-97df-4315-92f7-4929447b274c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:34.548341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c66051-97df-4315-92f7-4929447b274c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:34.548755 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c66051-97df-4315-92f7-4929447b274c None admin] PATH_INFO: `/v3/users/4abb99ab49734e918d15103010c2b0c4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:34.550271 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c66051-97df-4315-92f7-4929447b274c None admin] RBAC: Authorizing `identity:get_user(user_id=4abb99ab49734e918d15103010c2b0c4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:34.552085 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c66051-97df-4315-92f7-4929447b274c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:34.554282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 99/194] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:34 2022] GET /identity/v3/users/4abb99ab49734e918d15103010c2b0c4 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:34.560483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c66051-97df-4315-92f7-4929447b274c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:34.562095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a92808-4094-4528-9d9d-1c3b095102b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:34.563242 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a92808-4094-4528-9d9d-1c3b095102b5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:34.563919 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a92808-4094-4528-9d9d-1c3b095102b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:34.564479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a92808-4094-4528-9d9d-1c3b095102b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:34.691929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c66051-97df-4315-92f7-4929447b274c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:34.694261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a92808-4094-4528-9d9d-1c3b095102b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:34.697160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a92808-4094-4528-9d9d-1c3b095102b5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:34.762350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97425550-c47f-42c9-a57d-aa9201dcdf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:34.763001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97425550-c47f-42c9-a57d-aa9201dcdf97 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:34.763286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97425550-c47f-42c9-a57d-aa9201dcdf97 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:34.763563 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97425550-c47f-42c9-a57d-aa9201dcdf97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:34.801303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:34.813977 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:34.858482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97425550-c47f-42c9-a57d-aa9201dcdf97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:34.860407 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97425550-c47f-42c9-a57d-aa9201dcdf97 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:34.935616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 96/195] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:29:34.937087 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba59cb-a026-4b30-9bc8-b19a4f903a39 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:34.937675 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba59cb-a026-4b30-9bc8-b19a4f903a39 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:34.938014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba59cb-a026-4b30-9bc8-b19a4f903a39 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:34.939669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 97/196] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:34.943810 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba59cb-a026-4b30-9bc8-b19a4f903a39 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:35.023964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 100/197] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 10:29:35.030315 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26817bc2-4f46-4753-8714-db7879a44c38 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:35.031631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26817bc2-4f46-4753-8714-db7879a44c38 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:35.031911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26817bc2-4f46-4753-8714-db7879a44c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:35.053287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-26817bc2-4f46-4753-8714-db7879a44c38 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:35.054596 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-26817bc2-4f46-4753-8714-db7879a44c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:35.128402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:35.165094 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba59cb-a026-4b30-9bc8-b19a4f903a39 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:35.166770 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 101/198] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:35.168696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26817bc2-4f46-4753-8714-db7879a44c38 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:35.206376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b4d6f-a427-4665-8cf4-623426f618cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:35.207625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b4d6f-a427-4665-8cf4-623426f618cb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:35.207910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b4d6f-a427-4665-8cf4-623426f618cb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:35.208187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b4d6f-a427-4665-8cf4-623426f618cb None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:35.209437 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b4d6f-a427-4665-8cf4-623426f618cb None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:35.210965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b4d6f-a427-4665-8cf4-623426f618cb None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:35.213434 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 98/199] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:34 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:35.219427 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b4d6f-a427-4665-8cf4-623426f618cb None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:35.249447 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:35.341219 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b4d6f-a427-4665-8cf4-623426f618cb None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:35.371804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26817bc2-4f46-4753-8714-db7879a44c38 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:35.383775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2dd3d-f24f-4e5a-a67c-d8b452028c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:35.384467 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2dd3d-f24f-4e5a-a67c-d8b452028c26 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:35.384753 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2dd3d-f24f-4e5a-a67c-d8b452028c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:35.385082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2dd3d-f24f-4e5a-a67c-d8b452028c26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:35.385943 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2dd3d-f24f-4e5a-a67c-d8b452028c26 None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=d4db61efd90f48a6b63ae8fd15240329, user.id=4abb99ab49734e918d15103010c2b0c4) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:35.386253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2dd3d-f24f-4e5a-a67c-d8b452028c26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:35.387809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2dd3d-f24f-4e5a-a67c-d8b452028c26 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:35.394764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 99/200] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:35 2022] GET /identity/v3/role_assignments?user.id=4abb99ab49734e918d15103010c2b0c4&scope.project.id=d4db61efd90f48a6b63ae8fd15240329&role.id=01714797cab143e1a7a150c068fa9315 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:35.395854 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2dd3d-f24f-4e5a-a67c-d8b452028c26 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:35.475178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9924c-4a94-4461-a630-a554d678379c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:35.475824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9924c-4a94-4461-a630-a554d678379c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:35.476108 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9924c-4a94-4461-a630-a554d678379c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:35.476387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9924c-4a94-4461-a630-a554d678379c None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:35.479254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9924c-4a94-4461-a630-a554d678379c None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:35.480762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9924c-4a94-4461-a630-a554d678379c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:35.482841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 102/201] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:35 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:35.495139 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9924c-4a94-4461-a630-a554d678379c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:35.571927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:35.582201 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2dd3d-f24f-4e5a-a67c-d8b452028c26 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:35.617606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:35.625516 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7a5eb-8d4d-49a7-afc6-ea9f3b2f829a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:35.626372 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7a5eb-8d4d-49a7-afc6-ea9f3b2f829a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:35.626764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7a5eb-8d4d-49a7-afc6-ea9f3b2f829a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:35.627149 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7a5eb-8d4d-49a7-afc6-ea9f3b2f829a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:35.640098 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7a5eb-8d4d-49a7-afc6-ea9f3b2f829a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-00f7a5eb-8d4d-49a7-afc6-ea9f3b2f829a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:35.647066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:35.649014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:35.650750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:35.650750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:35.650750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:35.650750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:35.650750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:35.650750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:35.650750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:35.650750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:35.650750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:35.650750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:35.650750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:35.650750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:35.652058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 100/202] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:35.660640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7a5eb-8d4d-49a7-afc6-ea9f3b2f829a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:35.715617 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9924c-4a94-4461-a630-a554d678379c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:35.786364 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f3d16-89f1-4e06-921e-8794293bc0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:35.787004 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f3d16-89f1-4e06-921e-8794293bc0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:35.787287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f3d16-89f1-4e06-921e-8794293bc0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:35.787569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f3d16-89f1-4e06-921e-8794293bc0c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:35.788401 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f3d16-89f1-4e06-921e-8794293bc0c0 None admin] RBAC: Adding query filter params (role.id=cde4978cbb9648639ff720b5ca82886b, scope.project.id=d4db61efd90f48a6b63ae8fd15240329, user.id=4abb99ab49734e918d15103010c2b0c4) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:35.788717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f3d16-89f1-4e06-921e-8794293bc0c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:35.790247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f3d16-89f1-4e06-921e-8794293bc0c0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:35.804704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 103/203] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:35 2022] GET /identity/v3/role_assignments?user.id=4abb99ab49734e918d15103010c2b0c4&scope.project.id=d4db61efd90f48a6b63ae8fd15240329&role.id=cde4978cbb9648639ff720b5ca82886b => generated 259 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:35.869957 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7a5eb-8d4d-49a7-afc6-ea9f3b2f829a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:35.932024 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb534c-f020-4503-8cfe-a6e54b9bc7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:35.933133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb534c-f020-4503-8cfe-a6e54b9bc7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:35.933722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb534c-f020-4503-8cfe-a6e54b9bc7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:35.934341 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb534c-f020-4503-8cfe-a6e54b9bc7c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:35.935898 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb534c-f020-4503-8cfe-a6e54b9bc7c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:35.936511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb534c-f020-4503-8cfe-a6e54b9bc7c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:35.939215 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb534c-f020-4503-8cfe-a6e54b9bc7c5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:35.955638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 101/204] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:35.966675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f3d16-89f1-4e06-921e-8794293bc0c0 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:36.139119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f3d16-89f1-4e06-921e-8794293bc0c0 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:36.158252 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:36.178952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba13cc6-5c8b-4d8d-bd6c-fae58d3a4685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:36.179759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba13cc6-5c8b-4d8d-bd6c-fae58d3a4685 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:36.180067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba13cc6-5c8b-4d8d-bd6c-fae58d3a4685 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:36.180396 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba13cc6-5c8b-4d8d-bd6c-fae58d3a4685 None admin] PATH_INFO: `/v3/users/3d79674fdcd54f0fa0cf65c28d4bbdf8` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:36.181783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba13cc6-5c8b-4d8d-bd6c-fae58d3a4685 None admin] RBAC: Authorizing `identity:get_user(user_id=3d79674fdcd54f0fa0cf65c28d4bbdf8)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:36.183403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba13cc6-5c8b-4d8d-bd6c-fae58d3a4685 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:36.185386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 104/205] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:35 2022] GET /identity/v3/users/3d79674fdcd54f0fa0cf65c28d4bbdf8 => generated 289 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:36.191234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb534c-f020-4503-8cfe-a6e54b9bc7c5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:36.204786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:36.326932 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb534c-f020-4503-8cfe-a6e54b9bc7c5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:36.377506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f0397-4543-40f5-98b3-e195c4f1a1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:36.378271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f0397-4543-40f5-98b3-e195c4f1a1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:36.379103 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f0397-4543-40f5-98b3-e195c4f1a1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:36.379381 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f0397-4543-40f5-98b3-e195c4f1a1a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:36.477686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f0397-4543-40f5-98b3-e195c4f1a1a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:36.479247 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f0397-4543-40f5-98b3-e195c4f1a1a9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:36.494981 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:36.622385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 102/206] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:29:36.628225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba13cc6-5c8b-4d8d-bd6c-fae58d3a4685 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:36.711452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4ea2f-6edc-47d2-b08c-49a6459c55da None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:36.711793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4ea2f-6edc-47d2-b08c-49a6459c55da None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:36.712000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4ea2f-6edc-47d2-b08c-49a6459c55da None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:36.713766 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 103/207] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:36.719465 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f8dcd-5fa0-40d2-b724-234862ef4d24 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:36.719775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f8dcd-5fa0-40d2-b724-234862ef4d24 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:36.720014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f8dcd-5fa0-40d2-b724-234862ef4d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:36.745725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-af9f8dcd-5fa0-40d2-b724-234862ef4d24 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:36.746924 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-af9f8dcd-5fa0-40d2-b724-234862ef4d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:36.851206 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba13cc6-5c8b-4d8d-bd6c-fae58d3a4685 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:36.884538 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 104/208] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:36.921730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f8dcd-5fa0-40d2-b724-234862ef4d24 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:36.926610 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19c1ed-2cca-40e7-bd71-8094524bbdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:36.927288 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19c1ed-2cca-40e7-bd71-8094524bbdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:36.927568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19c1ed-2cca-40e7-bd71-8094524bbdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:36.927847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19c1ed-2cca-40e7-bd71-8094524bbdb9 None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:36.929178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19c1ed-2cca-40e7-bd71-8094524bbdb9 None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:36.932241 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19c1ed-2cca-40e7-bd71-8094524bbdb9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:36.934478 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 105/209] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:36 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:36.944512 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19c1ed-2cca-40e7-bd71-8094524bbdb9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:37.078155 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:37.165581 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f8dcd-5fa0-40d2-b724-234862ef4d24 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:37.172583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:37.230694 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c25b23-18e1-4dc0-adcb-1748f7c21346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:37.231381 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c25b23-18e1-4dc0-adcb-1748f7c21346 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:37.231663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c25b23-18e1-4dc0-adcb-1748f7c21346 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:37.231939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c25b23-18e1-4dc0-adcb-1748f7c21346 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:37.255372 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c25b23-18e1-4dc0-adcb-1748f7c21346 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-21c25b23-18e1-4dc0-adcb-1748f7c21346 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:37.261711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:37.262622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:37.263414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:37.263414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:37.263414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:37.263414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:37.263414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:37.263414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:37.263414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:37.263414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:37.263414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:37.263414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:37.263414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:37.263414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:37.268403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19c1ed-2cca-40e7-bd71-8094524bbdb9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:37.275666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 105/210] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:37.277318 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c25b23-18e1-4dc0-adcb-1748f7c21346 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:37.356509 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1119b7-4db7-4769-aa22-082bae2f8e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:37.357065 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1119b7-4db7-4769-aa22-082bae2f8e1f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:37.357349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1119b7-4db7-4769-aa22-082bae2f8e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:37.357651 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1119b7-4db7-4769-aa22-082bae2f8e1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:37.358501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1119b7-4db7-4769-aa22-082bae2f8e1f None admin] RBAC: Adding query filter params (role.id=66ef9c3aa98247eab1c1c726f2f6a5a5, scope.project.id=998a52ff5527415da541925c79fddaab, user.id=3d79674fdcd54f0fa0cf65c28d4bbdf8) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:37.358809 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1119b7-4db7-4769-aa22-082bae2f8e1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:37.360672 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1119b7-4db7-4769-aa22-082bae2f8e1f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:37.372384 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 106/211] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:36 2022] GET /identity/v3/role_assignments?user.id=3d79674fdcd54f0fa0cf65c28d4bbdf8&scope.project.id=998a52ff5527415da541925c79fddaab&role.id=66ef9c3aa98247eab1c1c726f2f6a5a5 => generated 608 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:37.405377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939fd1c-9895-48ca-ba5e-566a4d8cd98a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:37.406042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939fd1c-9895-48ca-ba5e-566a4d8cd98a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:37.406531 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939fd1c-9895-48ca-ba5e-566a4d8cd98a None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:37.408261 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 107/212] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:29:37.415249 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91b379-f911-414a-b721-88df550bd939 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:37.415553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91b379-f911-414a-b721-88df550bd939 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:37.415974 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91b379-f911-414a-b721-88df550bd939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:37.446497 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7e91b379-f911-414a-b721-88df550bd939 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:37.448034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7e91b379-f911-414a-b721-88df550bd939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:37.545893 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:37.557915 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c25b23-18e1-4dc0-adcb-1748f7c21346 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:37.636079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6e1ec-8f12-4e25-b694-01db58d3f0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:37.636746 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6e1ec-8f12-4e25-b694-01db58d3f0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:37.637108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6e1ec-8f12-4e25-b694-01db58d3f0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:37.637484 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6e1ec-8f12-4e25-b694-01db58d3f0a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:37.638327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6e1ec-8f12-4e25-b694-01db58d3f0a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:37.638763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6e1ec-8f12-4e25-b694-01db58d3f0a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:37.640477 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6e1ec-8f12-4e25-b694-01db58d3f0a7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:37.643969 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 108/213] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:37.660378 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 106/214] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:37.666573 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91b379-f911-414a-b721-88df550bd939 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:37.674497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6e1ec-8f12-4e25-b694-01db58d3f0a7 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:37.731369 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:37.858116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6e1ec-8f12-4e25-b694-01db58d3f0a7 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:37.869146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91b379-f911-414a-b721-88df550bd939 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:37.927259 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bd8d1-3b93-42a9-a9ea-66ba5025fac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:37.927838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bd8d1-3b93-42a9-a9ea-66ba5025fac1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:37.928119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bd8d1-3b93-42a9-a9ea-66ba5025fac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:37.928465 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bd8d1-3b93-42a9-a9ea-66ba5025fac1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:37.931996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900bd8d1-3b93-42a9-a9ea-66ba5025fac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:37.931996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900bd8d1-3b93-42a9-a9ea-66ba5025fac1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:37.935157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 107/215] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:29:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:37.940873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bd8d1-3b93-42a9-a9ea-66ba5025fac1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:37.941964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae601f-13a1-4bfb-8282-860acd21f0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:37.948025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae601f-13a1-4bfb-8282-860acd21f0dc None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:37.948913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae601f-13a1-4bfb-8282-860acd21f0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:37.949791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae601f-13a1-4bfb-8282-860acd21f0dc None admin] PATH_INFO: `/v3/users/a037d04644544ed58c9334d691d41237` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:37.982683 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae601f-13a1-4bfb-8282-860acd21f0dc None admin] RBAC: Authorizing `identity:get_user(user_id=a037d04644544ed58c9334d691d41237)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:37.989173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae601f-13a1-4bfb-8282-860acd21f0dc None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:37.992097 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 109/216] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:37 2022] GET /identity/v3/users/a037d04644544ed58c9334d691d41237 => generated 297 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:37.998182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:38.003377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae601f-13a1-4bfb-8282-860acd21f0dc None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:38.138202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae601f-13a1-4bfb-8282-860acd21f0dc None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:38.141932 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:38.147546 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bd8d1-3b93-42a9-a9ea-66ba5025fac1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:38.208825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e98b51-8c8b-4814-a2a0-372078a853de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:38.210389 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e98b51-8c8b-4814-a2a0-372078a853de None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:38.211175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e98b51-8c8b-4814-a2a0-372078a853de None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:38.211794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e98b51-8c8b-4814-a2a0-372078a853de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:38.217782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9fa2a-3232-4a5f-a153-d26390e9a493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:38.219203 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9fa2a-3232-4a5f-a153-d26390e9a493 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:38.219705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9fa2a-3232-4a5f-a153-d26390e9a493 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:38.220191 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9fa2a-3232-4a5f-a153-d26390e9a493 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:38.221524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9fa2a-3232-4a5f-a153-d26390e9a493 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:38.224309 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9fa2a-3232-4a5f-a153-d26390e9a493 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:38.260053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 108/217] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:29:37 2022] POST /identity/v3/users => generated 297 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:29:38.262318 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13728d1f-e240-4efa-a166-984fe7c07c5a None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:38.262950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13728d1f-e240-4efa-a166-984fe7c07c5a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:38.263456 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13728d1f-e240-4efa-a166-984fe7c07c5a None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:38.265996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 109/218] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:38.275940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289f699-4882-4ccd-8332-f02c3f3557b1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:38.276260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289f699-4882-4ccd-8332-f02c3f3557b1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:38.276650 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289f699-4882-4ccd-8332-f02c3f3557b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:38.307859 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4289f699-4882-4ccd-8332-f02c3f3557b1 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:38.309660 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4289f699-4882-4ccd-8332-f02c3f3557b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:38.310320 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e98b51-8c8b-4814-a2a0-372078a853de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:38.312218 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e98b51-8c8b-4814-a2a0-372078a853de None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:38.418028 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 110/219] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:38.435948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289f699-4882-4ccd-8332-f02c3f3557b1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:38.455646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 110/220] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:29:38.460029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e98b51-8c8b-4814-a2a0-372078a853de None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:38.466567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:38.488589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:38.591033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289f699-4882-4ccd-8332-f02c3f3557b1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:38.609274 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e98b51-8c8b-4814-a2a0-372078a853de None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:38.661326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791d32e-b332-4cbb-9103-e3561bb1a2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:38.662304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791d32e-b332-4cbb-9103-e3561bb1a2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:38.662589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791d32e-b332-4cbb-9103-e3561bb1a2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:38.662864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791d32e-b332-4cbb-9103-e3561bb1a2b6 None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:38.664107 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791d32e-b332-4cbb-9103-e3561bb1a2b6 None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:38.665690 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791d32e-b332-4cbb-9103-e3561bb1a2b6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:38.667573 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 111/221] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:38 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:38.670157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453683ee-0a1c-44ec-a15f-2d284c5fdd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:38.671092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453683ee-0a1c-44ec-a15f-2d284c5fdd62 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:38.671439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453683ee-0a1c-44ec-a15f-2d284c5fdd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:38.671790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453683ee-0a1c-44ec-a15f-2d284c5fdd62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:38.678311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791d32e-b332-4cbb-9103-e3561bb1a2b6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:38.696237 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453683ee-0a1c-44ec-a15f-2d284c5fdd62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-453683ee-0a1c-44ec-a15f-2d284c5fdd62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:38.701512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:38.702436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:38.703226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:38.703226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:38.703226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:38.703226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:38.703226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:38.703226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:38.703226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:38.703226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:38.703226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:38.703226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:38.703226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:38.703226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:38.703226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 111/222] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:29:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:29:38.709170 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453683ee-0a1c-44ec-a15f-2d284c5fdd62 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:38.794507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:38.807135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:38.835669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453683ee-0a1c-44ec-a15f-2d284c5fdd62 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:38.867937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791d32e-b332-4cbb-9103-e3561bb1a2b6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:38.879827 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903bec02-84f5-4d89-a717-80f89c727883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:38.881393 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bec02-84f5-4d89-a717-80f89c727883 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:38.881931 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bec02-84f5-4d89-a717-80f89c727883 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:38.882415 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bec02-84f5-4d89-a717-80f89c727883 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:38.883499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903bec02-84f5-4d89-a717-80f89c727883 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:38.884080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903bec02-84f5-4d89-a717-80f89c727883 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:38.885753 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903bec02-84f5-4d89-a717-80f89c727883 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:38.896922 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 112/223] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:38.902702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903bec02-84f5-4d89-a717-80f89c727883 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:38.936666 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0f912-ac04-4388-95b1-04429901a99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:38.937321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0f912-ac04-4388-95b1-04429901a99b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:38.937955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0f912-ac04-4388-95b1-04429901a99b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:38.937955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0f912-ac04-4388-95b1-04429901a99b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:38.938826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0f912-ac04-4388-95b1-04429901a99b None admin] RBAC: Adding query filter params (role.id=82be5cfedcb3407f8d14522ef87247ac, scope.project.id=d4db61efd90f48a6b63ae8fd15240329, user.id=a037d04644544ed58c9334d691d41237) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:38.939212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0f912-ac04-4388-95b1-04429901a99b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:38.940603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0f912-ac04-4388-95b1-04429901a99b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:38.947608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 112/224] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:38 2022] GET /identity/v3/role_assignments?user.id=a037d04644544ed58c9334d691d41237&scope.project.id=d4db61efd90f48a6b63ae8fd15240329&role.id=82be5cfedcb3407f8d14522ef87247ac => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:38.948833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf131c67-4a16-43a4-9310-52614dedbb5b None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:38.949137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf131c67-4a16-43a4-9310-52614dedbb5b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:38.949413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf131c67-4a16-43a4-9310-52614dedbb5b None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:38.955228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 113/225] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:38.957362 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afcf0cf-e9c4-471d-8b22-c02331622078 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:38.957682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afcf0cf-e9c4-471d-8b22-c02331622078 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:38.957954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afcf0cf-e9c4-471d-8b22-c02331622078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:38.997929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7afcf0cf-e9c4-471d-8b22-c02331622078 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:38.999147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7afcf0cf-e9c4-471d-8b22-c02331622078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:39.063052 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903bec02-84f5-4d89-a717-80f89c727883 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:39.149385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1eece2-6290-44e6-a2c4-17bd04ed3c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:39.150145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1eece2-6290-44e6-a2c4-17bd04ed3c2d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:39.150391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1eece2-6290-44e6-a2c4-17bd04ed3c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:39.150622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1eece2-6290-44e6-a2c4-17bd04ed3c2d None admin] PATH_INFO: `/v3/users/4abb99ab49734e918d15103010c2b0c4` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:39.152522 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1eece2-6290-44e6-a2c4-17bd04ed3c2d None admin] RBAC: Authorizing `identity:get_user(user_id=4abb99ab49734e918d15103010c2b0c4)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:39.154027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1eece2-6290-44e6-a2c4-17bd04ed3c2d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:39.156065 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 113/226] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:38 2022] GET /identity/v3/users/4abb99ab49734e918d15103010c2b0c4 => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:39.162333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 114/227] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:39.176063 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1eece2-6290-44e6-a2c4-17bd04ed3c2d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:39.187596 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:39.203784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afcf0cf-e9c4-471d-8b22-c02331622078 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:39.327180 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:39.369155 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1eece2-6290-44e6-a2c4-17bd04ed3c2d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:39.385590 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afcf0cf-e9c4-471d-8b22-c02331622078 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:39.426430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f9c58-2337-4910-a18c-e7befd73ab74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:39.427256 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f9c58-2337-4910-a18c-e7befd73ab74 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:39.427649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f9c58-2337-4910-a18c-e7befd73ab74 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:39.428033 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f9c58-2337-4910-a18c-e7befd73ab74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:39.441685 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f9c58-2337-4910-a18c-e7befd73ab74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-2c4f9c58-2337-4910-a18c-e7befd73ab74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:39.447399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:39.448303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:39.449086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:39.449086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:39.449086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:39.449086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:39.449086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:39.449086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:39.449086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:39.449086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:39.449086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:39.449086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:39.449086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:29:39.449086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:39.449086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 115/228] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:29:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:39.454763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f9c58-2337-4910-a18c-e7befd73ab74 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:39.462874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501cff19-5f30-4a5f-886d-86aaf513bbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:39.463820 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501cff19-5f30-4a5f-886d-86aaf513bbfd None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:39.464103 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501cff19-5f30-4a5f-886d-86aaf513bbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:39.464384 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501cff19-5f30-4a5f-886d-86aaf513bbfd None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:39.470423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501cff19-5f30-4a5f-886d-86aaf513bbfd None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:39.471921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501cff19-5f30-4a5f-886d-86aaf513bbfd None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:39.474128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 114/229] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:39 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:39.480984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501cff19-5f30-4a5f-886d-86aaf513bbfd None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:39.581579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f9c58-2337-4910-a18c-e7befd73ab74 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:39.617874 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:39.635945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb131ac5-d8ee-42c0-897d-a381ca0f86d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:39.636610 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb131ac5-d8ee-42c0-897d-a381ca0f86d4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:39.636900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb131ac5-d8ee-42c0-897d-a381ca0f86d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:39.637182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb131ac5-d8ee-42c0-897d-a381ca0f86d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:39.655204 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb131ac5-d8ee-42c0-897d-a381ca0f86d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:39.655586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb131ac5-d8ee-42c0-897d-a381ca0f86d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:39.657187 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb131ac5-d8ee-42c0-897d-a381ca0f86d4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:39.678007 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 116/230] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:29:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:39.689312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:39.698983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb131ac5-d8ee-42c0-897d-a381ca0f86d4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:39.782632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501cff19-5f30-4a5f-886d-86aaf513bbfd None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:39.877883 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037bec6-2cd9-457b-b41a-bf918590f73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:39.879558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037bec6-2cd9-457b-b41a-bf918590f73b None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:39.880140 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037bec6-2cd9-457b-b41a-bf918590f73b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:39.880655 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037bec6-2cd9-457b-b41a-bf918590f73b None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329/users/4abb99ab49734e918d15103010c2b0c4/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:39.884202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037bec6-2cd9-457b-b41a-bf918590f73b None admin] RBAC: Authorizing `identity:create_grant(project_id=d4db61efd90f48a6b63ae8fd15240329, user_id=4abb99ab49734e918d15103010c2b0c4, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:39.888511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037bec6-2cd9-457b-b41a-bf918590f73b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:39.913732 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 115/231] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:29:39 2022] PUT /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329/users/4abb99ab49734e918d15103010c2b0c4/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:29:39.955930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb131ac5-d8ee-42c0-897d-a381ca0f86d4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:40.023392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59934560-6b81-4dcf-95e1-e9b917453c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:40.024526 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934560-6b81-4dcf-95e1-e9b917453c51 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:40.025099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934560-6b81-4dcf-95e1-e9b917453c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:40.025618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934560-6b81-4dcf-95e1-e9b917453c51 None admin] PATH_INFO: `/v3/users/4abb99ab49734e918d15103010c2b0c4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:40.027998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59934560-6b81-4dcf-95e1-e9b917453c51 None admin] RBAC: Authorizing `identity:get_user(user_id=4abb99ab49734e918d15103010c2b0c4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:40.030661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59934560-6b81-4dcf-95e1-e9b917453c51 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:40.034713 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 117/232] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:39 2022] GET /identity/v3/users/4abb99ab49734e918d15103010c2b0c4 => generated 283 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:40.044225 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037bec6-2cd9-457b-b41a-bf918590f73b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:40.254994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037bec6-2cd9-457b-b41a-bf918590f73b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:40.296711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:40.306587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7e9d5-c4f7-4be3-aeee-56b094668175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:40.307365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7e9d5-c4f7-4be3-aeee-56b094668175 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:40.308023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7e9d5-c4f7-4be3-aeee-56b094668175 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:40.308776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7e9d5-c4f7-4be3-aeee-56b094668175 None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:40.313503 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7e9d5-c4f7-4be3-aeee-56b094668175 None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:40.313503 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7e9d5-c4f7-4be3-aeee-56b094668175 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:40.313503 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 116/233] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:40 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:40.320072 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59934560-6b81-4dcf-95e1-e9b917453c51 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:40.387793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:40.438390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59934560-6b81-4dcf-95e1-e9b917453c51 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:40.480694 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0779f15-bec8-4b12-b87d-061dee7d2cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:40.481432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0779f15-bec8-4b12-b87d-061dee7d2cae None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:40.481754 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0779f15-bec8-4b12-b87d-061dee7d2cae None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:40.482081 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0779f15-bec8-4b12-b87d-061dee7d2cae None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329/users/4abb99ab49734e918d15103010c2b0c4/roles/cde4978cbb9648639ff720b5ca82886b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:40.485203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0779f15-bec8-4b12-b87d-061dee7d2cae None admin] RBAC: Authorizing `identity:create_grant(project_id=d4db61efd90f48a6b63ae8fd15240329, user_id=4abb99ab49734e918d15103010c2b0c4, role_id=cde4978cbb9648639ff720b5ca82886b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:40.486817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0779f15-bec8-4b12-b87d-061dee7d2cae None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:40.515286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 118/234] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:29:40 2022] PUT /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329/users/4abb99ab49734e918d15103010c2b0c4/roles/cde4978cbb9648639ff720b5ca82886b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:29:40.974018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b9fae-d0cf-4566-a0e8-123e5755bdc1 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:40.974347 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b9fae-d0cf-4566-a0e8-123e5755bdc1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:40.974585 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b9fae-d0cf-4566-a0e8-123e5755bdc1 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:40.976256 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 117/235] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:40.982520 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a052d1a-a6a3-4275-bd13-0a183e37a5f6 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:40.983024 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a052d1a-a6a3-4275-bd13-0a183e37a5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:40.983098 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a052d1a-a6a3-4275-bd13-0a183e37a5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:41.006608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0a052d1a-a6a3-4275-bd13-0a183e37a5f6 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:41.007843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0a052d1a-a6a3-4275-bd13-0a183e37a5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:41.115233 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 119/236] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:41.147537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bd930-233f-4418-ab9b-8a6b8532de6b None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:41.148221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bd930-233f-4418-ab9b-8a6b8532de6b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:41.148991 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bd930-233f-4418-ab9b-8a6b8532de6b None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:41.153672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 118/237] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:41.158928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b0cee-808a-4ddb-b405-8cf5b7573135 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:41.159286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b0cee-808a-4ddb-b405-8cf5b7573135 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:41.159560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b0cee-808a-4ddb-b405-8cf5b7573135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:41.160511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bd930-233f-4418-ab9b-8a6b8532de6b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:41.212485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a78b0cee-808a-4ddb-b405-8cf5b7573135 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:41.218540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a78b0cee-808a-4ddb-b405-8cf5b7573135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:41.353719 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:41.398311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bd930-233f-4418-ab9b-8a6b8532de6b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:41.399039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 120/238] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:41.436750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b0cee-808a-4ddb-b405-8cf5b7573135 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:41.500830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d8aac-1ec0-4c96-986d-feb2f407b4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:41.502244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d8aac-1ec0-4c96-986d-feb2f407b4ec None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:41.504491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d8aac-1ec0-4c96-986d-feb2f407b4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:41.505036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d8aac-1ec0-4c96-986d-feb2f407b4ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:41.511380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d8aac-1ec0-4c96-986d-feb2f407b4ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:41.513333 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d8aac-1ec0-4c96-986d-feb2f407b4ec None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:41.518927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 119/239] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:29:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:41.525258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d8aac-1ec0-4c96-986d-feb2f407b4ec None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:41.640111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:41.650956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b0cee-808a-4ddb-b405-8cf5b7573135 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:41.707251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d8aac-1ec0-4c96-986d-feb2f407b4ec None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:41.711255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65a81c-7260-4e7e-a90a-de8c521c6b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:41.711926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65a81c-7260-4e7e-a90a-de8c521c6b1f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:41.712212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65a81c-7260-4e7e-a90a-de8c521c6b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:41.712488 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65a81c-7260-4e7e-a90a-de8c521c6b1f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:41.725721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65a81c-7260-4e7e-a90a-de8c521c6b1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-da65a81c-7260-4e7e-a90a-de8c521c6b1f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:41.730936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:41.736188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:41.740434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:41.740434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:41.740434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:41.740434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:41.740434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:41.740434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:41.740434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:41.740434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:41.740434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:41.740434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:41.740434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:41.740434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:41.740434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 121/240] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:41.740434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ba404-9763-4437-b718-09ffa4684bac None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:41.746220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ba404-9763-4437-b718-09ffa4684bac None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:41.746220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ba404-9763-4437-b718-09ffa4684bac None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:41.746220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 122/241] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:41.746220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5849d-9021-4926-b8ee-94617a244b17 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:41.746220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5849d-9021-4926-b8ee-94617a244b17 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:41.746220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5849d-9021-4926-b8ee-94617a244b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:41.765255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d6e5849d-9021-4926-b8ee-94617a244b17 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:41.766534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d6e5849d-9021-4926-b8ee-94617a244b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:41.790868 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469878d9-ba44-444b-aa51-8e8a98fda2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:41.792497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469878d9-ba44-444b-aa51-8e8a98fda2f0 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:41.793285 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469878d9-ba44-444b-aa51-8e8a98fda2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:41.794049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469878d9-ba44-444b-aa51-8e8a98fda2f0 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:41.795718 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469878d9-ba44-444b-aa51-8e8a98fda2f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:41.799278 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469878d9-ba44-444b-aa51-8e8a98fda2f0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:41.800290 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:41.845647 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 123/243] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:41.846257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 120/243] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:29:41 2022] POST /identity/v3/users => generated 293 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:29:41.846544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5849d-9021-4926-b8ee-94617a244b17 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:41.891175 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469878d9-ba44-444b-aa51-8e8a98fda2f0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:41.978025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5849d-9021-4926-b8ee-94617a244b17 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:42.019495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b427a6-7b3b-46b6-9760-a356d5bf1d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:42.020426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b427a6-7b3b-46b6-9760-a356d5bf1d52 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:42.020553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b427a6-7b3b-46b6-9760-a356d5bf1d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:42.020883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b427a6-7b3b-46b6-9760-a356d5bf1d52 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:42.021964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b427a6-7b3b-46b6-9760-a356d5bf1d52 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:42.021964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b427a6-7b3b-46b6-9760-a356d5bf1d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:42.023380 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b427a6-7b3b-46b6-9760-a356d5bf1d52 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:42.034352 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 124/244] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:42.046749 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b427a6-7b3b-46b6-9760-a356d5bf1d52 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:42.082263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:42.133442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469878d9-ba44-444b-aa51-8e8a98fda2f0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:42.183728 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:42.207545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8e91a-3687-4402-b429-173cd0eb4785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:42.208227 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8e91a-3687-4402-b429-173cd0eb4785 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:42.208439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b427a6-7b3b-46b6-9760-a356d5bf1d52 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:42.208567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8e91a-3687-4402-b429-173cd0eb4785 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:42.208769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8e91a-3687-4402-b429-173cd0eb4785 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:42.224151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8e91a-3687-4402-b429-173cd0eb4785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-71b8e91a-3687-4402-b429-173cd0eb4785 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:42.232543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:42.233554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:42.234797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:42.234797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:42.234797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:42.234797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:42.234797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:42.234797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:42.234797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:42.234797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:42.234797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:42.234797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:42.234797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:42.234797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:42.234797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 121/245] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:42.241050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8e91a-3687-4402-b429-173cd0eb4785 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:42.295365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da63283-5e32-49fd-807a-a804bb88dedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:42.296618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da63283-5e32-49fd-807a-a804bb88dedf None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:42.297331 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da63283-5e32-49fd-807a-a804bb88dedf None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:42.297945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da63283-5e32-49fd-807a-a804bb88dedf None admin] PATH_INFO: `/v3/users/7f58e6c51efb409cb64d3c907f8c7998` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:42.332578 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da63283-5e32-49fd-807a-a804bb88dedf None admin] RBAC: Authorizing `identity:get_user(user_id=7f58e6c51efb409cb64d3c907f8c7998)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:42.335385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da63283-5e32-49fd-807a-a804bb88dedf None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:42.337694 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 125/246] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:42 2022] GET /identity/v3/users/7f58e6c51efb409cb64d3c907f8c7998 => generated 297 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:42.344069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da63283-5e32-49fd-807a-a804bb88dedf None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:42.417737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8e91a-3687-4402-b429-173cd0eb4785 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:42.481686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:42.488357 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d8b27-3016-4bb5-8f6c-4fa91f2e6698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:42.489020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d8b27-3016-4bb5-8f6c-4fa91f2e6698 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:42.489306 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d8b27-3016-4bb5-8f6c-4fa91f2e6698 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:42.489614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d8b27-3016-4bb5-8f6c-4fa91f2e6698 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:42.490913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d8b27-3016-4bb5-8f6c-4fa91f2e6698 None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:42.490913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d8b27-3016-4bb5-8f6c-4fa91f2e6698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:42.492533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d8b27-3016-4bb5-8f6c-4fa91f2e6698 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:42.506081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 122/247] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:42.507890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da63283-5e32-49fd-807a-a804bb88dedf None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:42.512021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d8b27-3016-4bb5-8f6c-4fa91f2e6698 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:42.558874 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f46b9-e619-45de-8e2d-22836b72c0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:42.559887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f46b9-e619-45de-8e2d-22836b72c0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:42.560365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f46b9-e619-45de-8e2d-22836b72c0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:42.560850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f46b9-e619-45de-8e2d-22836b72c0e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:42.596998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:42.721705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f46b9-e619-45de-8e2d-22836b72c0e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:42.723579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f46b9-e619-45de-8e2d-22836b72c0e1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:42.729515 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d8b27-3016-4bb5-8f6c-4fa91f2e6698 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:42.812810 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5189e-54c4-4522-b765-9de7c37d60a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:42.814475 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5189e-54c4-4522-b765-9de7c37d60a3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:42.814989 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5189e-54c4-4522-b765-9de7c37d60a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:42.815771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5189e-54c4-4522-b765-9de7c37d60a3 None admin] PATH_INFO: `/v3/users/a037d04644544ed58c9334d691d41237` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:42.819088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5189e-54c4-4522-b765-9de7c37d60a3 None admin] RBAC: Authorizing `identity:get_user(user_id=a037d04644544ed58c9334d691d41237)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:42.821951 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5189e-54c4-4522-b765-9de7c37d60a3 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:42.826615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 123/248] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:42 2022] GET /identity/v3/users/a037d04644544ed58c9334d691d41237 => generated 297 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:42.829336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87583b0a-7892-4eed-a9fc-0a73da71df98 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:42.829874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87583b0a-7892-4eed-a9fc-0a73da71df98 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:42.830368 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87583b0a-7892-4eed-a9fc-0a73da71df98 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:42.833895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 124/249] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:42.836245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87583b0a-7892-4eed-a9fc-0a73da71df98 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:42.939730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:42.982832 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 126/250] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 10:29:42.984178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05555018-00f6-4537-8077-e779c92fa401 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:42.984473 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05555018-00f6-4537-8077-e779c92fa401 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:42.984752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05555018-00f6-4537-8077-e779c92fa401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:43.025984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-05555018-00f6-4537-8077-e779c92fa401 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:43.027270 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-05555018-00f6-4537-8077-e779c92fa401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:43.070066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87583b0a-7892-4eed-a9fc-0a73da71df98 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:43.103232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:43.151954 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737613e-986e-4b9b-9f8a-eb22faca5df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:43.153535 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737613e-986e-4b9b-9f8a-eb22faca5df4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:43.154236 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737613e-986e-4b9b-9f8a-eb22faca5df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:43.154910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737613e-986e-4b9b-9f8a-eb22faca5df4 None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:43.157391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737613e-986e-4b9b-9f8a-eb22faca5df4 None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:43.161553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737613e-986e-4b9b-9f8a-eb22faca5df4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:43.166263 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 125/251] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:42 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:43.169764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737613e-986e-4b9b-9f8a-eb22faca5df4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:43.224664 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 127/252] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:43.226671 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05555018-00f6-4537-8077-e779c92fa401 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:43.413441 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737613e-986e-4b9b-9f8a-eb22faca5df4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:43.418642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05555018-00f6-4537-8077-e779c92fa401 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:43.459402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:43.460023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08926370-7842-46eb-9846-286baa7fa79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:43.460926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08926370-7842-46eb-9846-286baa7fa79b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:43.461871 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08926370-7842-46eb-9846-286baa7fa79b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:43.462162 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08926370-7842-46eb-9846-286baa7fa79b None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:43.464693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08926370-7842-46eb-9846-286baa7fa79b None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:43.467793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08926370-7842-46eb-9846-286baa7fa79b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:43.473622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 126/253] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:43 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:43.478122 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08926370-7842-46eb-9846-286baa7fa79b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:43.511055 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ccb71-864b-4b46-8e87-e1d4e1540ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:43.513756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ccb71-864b-4b46-8e87-e1d4e1540ef9 None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:43.514525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ccb71-864b-4b46-8e87-e1d4e1540ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:43.515586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ccb71-864b-4b46-8e87-e1d4e1540ef9 None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329/users/a037d04644544ed58c9334d691d41237/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:43.521938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ccb71-864b-4b46-8e87-e1d4e1540ef9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4db61efd90f48a6b63ae8fd15240329, user_id=a037d04644544ed58c9334d691d41237, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:43.526138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ccb71-864b-4b46-8e87-e1d4e1540ef9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:43.552472 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:43.553901 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 128/254] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:29:43 2022] PUT /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329/users/a037d04644544ed58c9334d691d41237/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:29:43.556893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e02299-fdc3-4ca5-8146-b161077f556d None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:43.557267 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e02299-fdc3-4ca5-8146-b161077f556d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:43.557451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e02299-fdc3-4ca5-8146-b161077f556d None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:43.559081 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 129/255] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:43.561747 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e02299-fdc3-4ca5-8146-b161077f556d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:43.663567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08926370-7842-46eb-9846-286baa7fa79b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:43.709364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af125141-cc2d-41f7-b159-7aa4b4f950ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:43.710161 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af125141-cc2d-41f7-b159-7aa4b4f950ca None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:43.710513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af125141-cc2d-41f7-b159-7aa4b4f950ca None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:43.710796 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af125141-cc2d-41f7-b159-7aa4b4f950ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:43.738884 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af125141-cc2d-41f7-b159-7aa4b4f950ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-af125141-cc2d-41f7-b159-7aa4b4f950ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:43.745033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:43.746014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:43.746803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:43.746803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:43.746803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:43.746803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:43.746803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:43.746803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:43.746803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:43.746803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:43.746803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:43.746803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:43.746803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:29:43.746803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:43.754184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 127/256] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:29:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:43.755874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a796ebd-0834-43ea-8035-586d4015f183 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:43.756165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a796ebd-0834-43ea-8035-586d4015f183 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:43.756450 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a796ebd-0834-43ea-8035-586d4015f183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:43.772703 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e02299-fdc3-4ca5-8146-b161077f556d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:43.803431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8a796ebd-0834-43ea-8035-586d4015f183 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:43.805548 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8a796ebd-0834-43ea-8035-586d4015f183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:43.866218 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e1f76-7bb7-4c1c-89b6-4cc0b118fee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:43.866872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e1f76-7bb7-4c1c-89b6-4cc0b118fee5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:43.867166 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e1f76-7bb7-4c1c-89b6-4cc0b118fee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:43.867446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e1f76-7bb7-4c1c-89b6-4cc0b118fee5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:43.868402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e1f76-7bb7-4c1c-89b6-4cc0b118fee5 None admin] RBAC: Adding query filter params (role.id=66ef9c3aa98247eab1c1c726f2f6a5a5, scope.project.id=d4db61efd90f48a6b63ae8fd15240329, user.id=7f58e6c51efb409cb64d3c907f8c7998) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:43.868726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e1f76-7bb7-4c1c-89b6-4cc0b118fee5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:43.870347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e1f76-7bb7-4c1c-89b6-4cc0b118fee5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:43.880219 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 130/257] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:43 2022] GET /identity/v3/role_assignments?user.id=7f58e6c51efb409cb64d3c907f8c7998&scope.project.id=d4db61efd90f48a6b63ae8fd15240329&role.id=66ef9c3aa98247eab1c1c726f2f6a5a5 => generated 259 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:43.882203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e1f76-7bb7-4c1c-89b6-4cc0b118fee5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:44.011941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 128/258] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:44.012466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:44.058606 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a796ebd-0834-43ea-8035-586d4015f183 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:44.161493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e1f76-7bb7-4c1c-89b6-4cc0b118fee5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:44.170499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:44.246524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a796ebd-0834-43ea-8035-586d4015f183 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:44.290269 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe832f0-69b9-4071-b866-665ed058c385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:44.291409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe832f0-69b9-4071-b866-665ed058c385 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:44.292549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe832f0-69b9-4071-b866-665ed058c385 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:44.293182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe832f0-69b9-4071-b866-665ed058c385 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:44.294682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe832f0-69b9-4071-b866-665ed058c385 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:44.296805 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe832f0-69b9-4071-b866-665ed058c385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:44.298883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe832f0-69b9-4071-b866-665ed058c385 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:44.302916 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac5770-63f6-4be3-a931-264a33831da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:44.303846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac5770-63f6-4be3-a931-264a33831da5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:44.304311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac5770-63f6-4be3-a931-264a33831da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:44.304763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac5770-63f6-4be3-a931-264a33831da5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:44.319863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 131/259] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:44.327685 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac5770-63f6-4be3-a931-264a33831da5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:44.329110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe832f0-69b9-4071-b866-665ed058c385 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-d1ac5770-63f6-4be3-a931-264a33831da5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:44.340428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:44.341419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:44.342257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:44.342257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:44.342257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:44.342257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:44.342257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:44.342257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:44.342257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:44.342257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:44.342257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:44.342257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:44.342257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:29:44.342257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:44.355485 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 129/260] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:29:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:29:44.371973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac5770-63f6-4be3-a931-264a33831da5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:44.464416 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe832f0-69b9-4071-b866-665ed058c385 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:44.466887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:44.518517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c049905-8e0f-4bc3-875d-6c6960c63451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:44.519567 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c049905-8e0f-4bc3-875d-6c6960c63451 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:44.520158 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c049905-8e0f-4bc3-875d-6c6960c63451 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:44.521528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c049905-8e0f-4bc3-875d-6c6960c63451 None admin] PATH_INFO: `/v3/users/4abb99ab49734e918d15103010c2b0c4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:44.523334 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c049905-8e0f-4bc3-875d-6c6960c63451 None admin] RBAC: Authorizing `identity:get_user(user_id=4abb99ab49734e918d15103010c2b0c4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:44.526255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c049905-8e0f-4bc3-875d-6c6960c63451 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:44.529978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 132/261] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:44 2022] GET /identity/v3/users/4abb99ab49734e918d15103010c2b0c4 => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:44.540952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c049905-8e0f-4bc3-875d-6c6960c63451 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:44.657289 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:44.664276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac5770-63f6-4be3-a931-264a33831da5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:44.742975 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06289fd9-15d6-4fcd-8a80-9501533d0d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:44.743640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06289fd9-15d6-4fcd-8a80-9501533d0d5e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:44.743923 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06289fd9-15d6-4fcd-8a80-9501533d0d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:44.744202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06289fd9-15d6-4fcd-8a80-9501533d0d5e None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:44.744952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06289fd9-15d6-4fcd-8a80-9501533d0d5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:44.745262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06289fd9-15d6-4fcd-8a80-9501533d0d5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:44.746743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06289fd9-15d6-4fcd-8a80-9501533d0d5e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:44.758602 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 130/262] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:29:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:44.765080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06289fd9-15d6-4fcd-8a80-9501533d0d5e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:44.811872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c049905-8e0f-4bc3-875d-6c6960c63451 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:44.883339 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de924d00-7ebc-4ad2-ad46-4f2edcdb5961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:44.884388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de924d00-7ebc-4ad2-ad46-4f2edcdb5961 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:44.884876 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de924d00-7ebc-4ad2-ad46-4f2edcdb5961 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:44.885355 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de924d00-7ebc-4ad2-ad46-4f2edcdb5961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:44.949188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:44.988706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de924d00-7ebc-4ad2-ad46-4f2edcdb5961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:44.990234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de924d00-7ebc-4ad2-ad46-4f2edcdb5961 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:45.014296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06289fd9-15d6-4fcd-8a80-9501533d0d5e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:45.134324 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8edb9-37fa-4c43-b1c3-edb9f00804f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:45.134933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8edb9-37fa-4c43-b1c3-edb9f00804f4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:45.135227 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8edb9-37fa-4c43-b1c3-edb9f00804f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:45.135507 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8edb9-37fa-4c43-b1c3-edb9f00804f4 None admin] PATH_INFO: `/v3/users/4abb99ab49734e918d15103010c2b0c4` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:45.139615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8edb9-37fa-4c43-b1c3-edb9f00804f4 None admin] RBAC: Authorizing `identity:get_user(user_id=4abb99ab49734e918d15103010c2b0c4)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:45.141123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8edb9-37fa-4c43-b1c3-edb9f00804f4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:45.143216 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 131/263] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:44 2022] GET /identity/v3/users/4abb99ab49734e918d15103010c2b0c4 => generated 283 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:45.157076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8edb9-37fa-4c43-b1c3-edb9f00804f4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:45.179669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 133/264] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:29:45.188323 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de924d00-7ebc-4ad2-ad46-4f2edcdb5961 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:45.240723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:45.390088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:45.431275 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8edb9-37fa-4c43-b1c3-edb9f00804f4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:45.461680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de924d00-7ebc-4ad2-ad46-4f2edcdb5961 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:45.526769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5361a-f050-497a-b64e-377c4bdf7ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:45.527424 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5361a-f050-497a-b64e-377c4bdf7ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:45.527712 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5361a-f050-497a-b64e-377c4bdf7ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:45.527996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5361a-f050-497a-b64e-377c4bdf7ff1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:45.551874 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1efcd-7fec-4e50-bf59-678fe6441cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:45.552548 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1efcd-7fec-4e50-bf59-678fe6441cce None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:45.552835 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1efcd-7fec-4e50-bf59-678fe6441cce None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:45.553112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1efcd-7fec-4e50-bf59-678fe6441cce None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:45.556601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1efcd-7fec-4e50-bf59-678fe6441cce None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:45.558120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1efcd-7fec-4e50-bf59-678fe6441cce None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:45.561520 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 134/265] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:45 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:45.562773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bcbd9-c8f4-428f-be49-084dc03266fe None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:45.563059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bcbd9-c8f4-428f-be49-084dc03266fe None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:45.563338 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bcbd9-c8f4-428f-be49-084dc03266fe None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:45.564611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 135/266] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:45.574253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bcbd9-c8f4-428f-be49-084dc03266fe None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:45.648576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5361a-f050-497a-b64e-377c4bdf7ff1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:45.650205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5361a-f050-497a-b64e-377c4bdf7ff1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:45.683131 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:45.765910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 132/267] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:29:45.767238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fd05d-9022-407e-9f94-5acfc827766c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:45.767538 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fd05d-9022-407e-9f94-5acfc827766c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:45.767814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fd05d-9022-407e-9f94-5acfc827766c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:45.801550 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-090fd05d-9022-407e-9f94-5acfc827766c None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:45.802802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-090fd05d-9022-407e-9f94-5acfc827766c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:45.812318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bcbd9-c8f4-428f-be49-084dc03266fe None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:45.882642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1b905-7ddc-4e59-8f4c-c7d8df6e30da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:45.883297 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1b905-7ddc-4e59-8f4c-c7d8df6e30da None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:45.883587 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1b905-7ddc-4e59-8f4c-c7d8df6e30da None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:45.883867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1b905-7ddc-4e59-8f4c-c7d8df6e30da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:45.884695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1b905-7ddc-4e59-8f4c-c7d8df6e30da None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=d4db61efd90f48a6b63ae8fd15240329, user.id=4abb99ab49734e918d15103010c2b0c4) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:45.885006 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1b905-7ddc-4e59-8f4c-c7d8df6e30da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:45.886529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1b905-7ddc-4e59-8f4c-c7d8df6e30da None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:45.906277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 136/268] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:45 2022] GET /identity/v3/role_assignments?user.id=4abb99ab49734e918d15103010c2b0c4&scope.project.id=d4db61efd90f48a6b63ae8fd15240329&role.id=01714797cab143e1a7a150c068fa9315 => generated 608 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:45.906900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1b905-7ddc-4e59-8f4c-c7d8df6e30da None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:45.916120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:45.982645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 133/269] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:46.030007 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fd05d-9022-407e-9f94-5acfc827766c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:46.215648 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:46.247915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1b905-7ddc-4e59-8f4c-c7d8df6e30da None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:46.299442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fd05d-9022-407e-9f94-5acfc827766c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:46.308183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20048774-61c3-4cea-930b-f9dd6a8bd255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:46.309667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20048774-61c3-4cea-930b-f9dd6a8bd255 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:46.310358 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20048774-61c3-4cea-930b-f9dd6a8bd255 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:46.311005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20048774-61c3-4cea-930b-f9dd6a8bd255 None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:46.314034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20048774-61c3-4cea-930b-f9dd6a8bd255 None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:46.318087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20048774-61c3-4cea-930b-f9dd6a8bd255 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:46.322233 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 137/270] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:45 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:46.324757 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc846ce6-9672-4d78-97bd-3400776d1038 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:46.326844 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc846ce6-9672-4d78-97bd-3400776d1038 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:46.328111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc846ce6-9672-4d78-97bd-3400776d1038 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:46.331156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 138/271] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:46.332844 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc846ce6-9672-4d78-97bd-3400776d1038 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:46.373991 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d0d52-c90c-4044-948e-fd5e96e3b1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:46.375771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d0d52-c90c-4044-948e-fd5e96e3b1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:46.376633 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d0d52-c90c-4044-948e-fd5e96e3b1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:46.377405 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d0d52-c90c-4044-948e-fd5e96e3b1f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:46.398697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d0d52-c90c-4044-948e-fd5e96e3b1f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-0e8d0d52-c90c-4044-948e-fd5e96e3b1f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:46.409410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:46.411736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:46.413298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:46.413298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:46.413298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:46.413298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:46.413298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:46.413298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:46.413298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:46.413298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:46.413298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:46.413298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:46.413298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:46.413298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:46.417789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 134/272] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:46.419790 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:46.420781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb190747-734f-4621-9b1a-425ac6070ccc None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:46.421626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb190747-734f-4621-9b1a-425ac6070ccc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:46.422407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb190747-734f-4621-9b1a-425ac6070ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:46.456040 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-eb190747-734f-4621-9b1a-425ac6070ccc None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:46.459205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-eb190747-734f-4621-9b1a-425ac6070ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:46.488164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc846ce6-9672-4d78-97bd-3400776d1038 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:46.565564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6edc6e-9a9b-476c-9493-1c749a941728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:46.566852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6edc6e-9a9b-476c-9493-1c749a941728 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:46.567447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6edc6e-9a9b-476c-9493-1c749a941728 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:46.568042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6edc6e-9a9b-476c-9493-1c749a941728 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:46.569812 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6edc6e-9a9b-476c-9493-1c749a941728 None admin] RBAC: Adding query filter params (role.id=cde4978cbb9648639ff720b5ca82886b, scope.project.id=d4db61efd90f48a6b63ae8fd15240329, user.id=4abb99ab49734e918d15103010c2b0c4) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:46.570474 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6edc6e-9a9b-476c-9493-1c749a941728 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:46.573512 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6edc6e-9a9b-476c-9493-1c749a941728 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:46.589844 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 139/273] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:46 2022] GET /identity/v3/role_assignments?user.id=4abb99ab49734e918d15103010c2b0c4&scope.project.id=d4db61efd90f48a6b63ae8fd15240329&role.id=cde4978cbb9648639ff720b5ca82886b => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:46.593444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6edc6e-9a9b-476c-9493-1c749a941728 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:46.611530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 135/274] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:46.661591 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb190747-734f-4621-9b1a-425ac6070ccc None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:46.759658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:46.864567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb190747-734f-4621-9b1a-425ac6070ccc None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:46.899555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6edc6e-9a9b-476c-9493-1c749a941728 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:46.935353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2b137-0def-45d7-a9e5-b471d80f63e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:46.937596 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2b137-0def-45d7-a9e5-b471d80f63e8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:46.938252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2b137-0def-45d7-a9e5-b471d80f63e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:46.938888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2b137-0def-45d7-a9e5-b471d80f63e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:46.942020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2b137-0def-45d7-a9e5-b471d80f63e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:46.945058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2b137-0def-45d7-a9e5-b471d80f63e8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:46.948593 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 136/275] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:29:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:46.953077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f74f94-fd19-4944-8387-dac01abb7234 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:46.953077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f74f94-fd19-4944-8387-dac01abb7234 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:46.953077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f74f94-fd19-4944-8387-dac01abb7234 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:46.955547 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 137/276] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:46.956320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f74f94-fd19-4944-8387-dac01abb7234 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:46.973874 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e03cde-4651-4d6d-821d-0de972af5c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:46.974994 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e03cde-4651-4d6d-821d-0de972af5c92 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:46.975507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e03cde-4651-4d6d-821d-0de972af5c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:46.976039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e03cde-4651-4d6d-821d-0de972af5c92 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:46.977034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e03cde-4651-4d6d-821d-0de972af5c92 None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:46.977633 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e03cde-4651-4d6d-821d-0de972af5c92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:46.979337 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e03cde-4651-4d6d-821d-0de972af5c92 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:46.981359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:46.991502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 140/277] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:46.997606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162e288-b6c6-4847-9159-511b73df9a2a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:46.997906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162e288-b6c6-4847-9159-511b73df9a2a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:46.998155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162e288-b6c6-4847-9159-511b73df9a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:47.019488 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5162e288-b6c6-4847-9159-511b73df9a2a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:47.020684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5162e288-b6c6-4847-9159-511b73df9a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:47.111650 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f74f94-fd19-4944-8387-dac01abb7234 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:47.132666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:47.137552 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 141/278] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:47.137552 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de05ea9-9d6a-4684-9e88-8d1d14f73fe1 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:47.137552 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de05ea9-9d6a-4684-9e88-8d1d14f73fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:47.137990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de05ea9-9d6a-4684-9e88-8d1d14f73fe1 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:47.140587 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 142/279] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:47.141760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de05ea9-9d6a-4684-9e88-8d1d14f73fe1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:47.167358 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb368b7-9938-4682-9f30-37430864acb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:47.168414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb368b7-9938-4682-9f30-37430864acb8 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:47.168902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb368b7-9938-4682-9f30-37430864acb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:47.169383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb368b7-9938-4682-9f30-37430864acb8 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:47.170695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb368b7-9938-4682-9f30-37430864acb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:47.172979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb368b7-9938-4682-9f30-37430864acb8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:47.203654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 138/280] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:29:46 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:29:47.204927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0b919-f84b-4cb2-9837-9a958019b798 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:47.205219 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0b919-f84b-4cb2-9837-9a958019b798 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:47.205512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0b919-f84b-4cb2-9837-9a958019b798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:47.228187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-30f0b919-f84b-4cb2-9837-9a958019b798 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:47.229564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-30f0b919-f84b-4cb2-9837-9a958019b798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:47.298025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de05ea9-9d6a-4684-9e88-8d1d14f73fe1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:47.345502 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 139/281] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:47.345502 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0b919-f84b-4cb2-9837-9a958019b798 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:47.356030 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:47.366221 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18e2e0-5a38-41e8-a5b7-e8a46ad7beca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:47.367299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18e2e0-5a38-41e8-a5b7-e8a46ad7beca None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:47.367643 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18e2e0-5a38-41e8-a5b7-e8a46ad7beca None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:47.367937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18e2e0-5a38-41e8-a5b7-e8a46ad7beca None admin] PATH_INFO: `/v3/users/f8ba9a843c0442ca81a2cf3a524e5fa3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:47.389119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be18e2e0-5a38-41e8-a5b7-e8a46ad7beca None admin] RBAC: Authorizing `identity:get_user(user_id=f8ba9a843c0442ca81a2cf3a524e5fa3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:47.390653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be18e2e0-5a38-41e8-a5b7-e8a46ad7beca None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:47.392627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 143/282] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:47 2022] GET /identity/v3/users/f8ba9a843c0442ca81a2cf3a524e5fa3 => generated 293 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:47.394771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18e2e0-5a38-41e8-a5b7-e8a46ad7beca None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:47.529658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0b919-f84b-4cb2-9837-9a958019b798 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:47.537513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:47.592766 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18e2e0-5a38-41e8-a5b7-e8a46ad7beca None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:47.615320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cb6c7-58b8-4e39-ba09-a4fcb63eba39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:47.616790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cb6c7-58b8-4e39-ba09-a4fcb63eba39 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:47.617516 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cb6c7-58b8-4e39-ba09-a4fcb63eba39 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:47.618191 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cb6c7-58b8-4e39-ba09-a4fcb63eba39 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:47.637798 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cb6c7-58b8-4e39-ba09-a4fcb63eba39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:47.639285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93514e-5a52-4d9d-a76a-2cf56eb3d500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:47.640038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93514e-5a52-4d9d-a76a-2cf56eb3d500 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:47.640385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93514e-5a52-4d9d-a76a-2cf56eb3d500 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:47.640728 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93514e-5a52-4d9d-a76a-2cf56eb3d500 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:47.653882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93514e-5a52-4d9d-a76a-2cf56eb3d500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-134cb6c7-58b8-4e39-ba09-a4fcb63eba39 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:47.655889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:47.657878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:47.659802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:47.659802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:47.659802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:47.659802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:47.659802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:47.659802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:47.659802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:47.659802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:47.659802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:47.659802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:47.659802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:47.659802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:47.659802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 140/283] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:47.659802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cb6c7-58b8-4e39-ba09-a4fcb63eba39 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-3d93514e-5a52-4d9d-a76a-2cf56eb3d500 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:47.667423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:47.669372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:47.671425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:47.671425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:47.671425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:47.671425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:47.671425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:47.671425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:47.671425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:47.671425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:47.671425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:47.671425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:47.671425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:47.671425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:47.671425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 144/284] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:47.671425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93514e-5a52-4d9d-a76a-2cf56eb3d500 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:47.738574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:47.789497 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93514e-5a52-4d9d-a76a-2cf56eb3d500 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:47.829777 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd6dca-54f5-4210-84ad-29b78cc38179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:47.830592 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd6dca-54f5-4210-84ad-29b78cc38179 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:47.830985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd6dca-54f5-4210-84ad-29b78cc38179 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:47.831390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd6dca-54f5-4210-84ad-29b78cc38179 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:47.832246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd6dca-54f5-4210-84ad-29b78cc38179 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:47.832646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd6dca-54f5-4210-84ad-29b78cc38179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:47.834302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd6dca-54f5-4210-84ad-29b78cc38179 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:47.849692 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 145/285] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:47.850935 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd6dca-54f5-4210-84ad-29b78cc38179 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:47.855973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cb6c7-58b8-4e39-ba09-a4fcb63eba39 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:47.919157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39238ddf-fb92-44b7-b17f-cf190e30fc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:47.920565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39238ddf-fb92-44b7-b17f-cf190e30fc98 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:47.921062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39238ddf-fb92-44b7-b17f-cf190e30fc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:47.921794 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39238ddf-fb92-44b7-b17f-cf190e30fc98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:47.924126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39238ddf-fb92-44b7-b17f-cf190e30fc98 None admin] RBAC: Adding query filter params (role.id=82be5cfedcb3407f8d14522ef87247ac, user.id=f8ba9a843c0442ca81a2cf3a524e5fa3, scope.system=all) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:47.924675 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39238ddf-fb92-44b7-b17f-cf190e30fc98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:47.929249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39238ddf-fb92-44b7-b17f-cf190e30fc98 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:47.946222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 141/286] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Aug 3 10:29:47 2022] GET /identity/v3/role_assignments?user.id=f8ba9a843c0442ca81a2cf3a524e5fa3&scope.system=all&role.id=82be5cfedcb3407f8d14522ef87247ac => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:47.948525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39238ddf-fb92-44b7-b17f-cf190e30fc98 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:47.980249 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd6dca-54f5-4210-84ad-29b78cc38179 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:48.003957 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:48.021377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b60fc-481c-4543-9872-51edebde9d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:48.022035 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b60fc-481c-4543-9872-51edebde9d24 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:48.022420 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b60fc-481c-4543-9872-51edebde9d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:48.022800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b60fc-481c-4543-9872-51edebde9d24 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:48.023675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b60fc-481c-4543-9872-51edebde9d24 None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:48.024092 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b60fc-481c-4543-9872-51edebde9d24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:48.025828 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b60fc-481c-4543-9872-51edebde9d24 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:48.037103 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 146/287] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:48.037518 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:48.044122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b60fc-481c-4543-9872-51edebde9d24 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:48.102721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39238ddf-fb92-44b7-b17f-cf190e30fc98 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:48.200362 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507643d-427f-461b-a2a6-09ca8377d166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:48.201546 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507643d-427f-461b-a2a6-09ca8377d166 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:48.201991 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507643d-427f-461b-a2a6-09ca8377d166 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:48.202441 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507643d-427f-461b-a2a6-09ca8377d166 None admin] PATH_INFO: `/v3/users/7f58e6c51efb409cb64d3c907f8c7998` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:48.204289 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507643d-427f-461b-a2a6-09ca8377d166 None admin] RBAC: Authorizing `identity:get_user(user_id=7f58e6c51efb409cb64d3c907f8c7998)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:48.209754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507643d-427f-461b-a2a6-09ca8377d166 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:48.223942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 142/288] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:47 2022] GET /identity/v3/users/7f58e6c51efb409cb64d3c907f8c7998 => generated 297 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:48.231418 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507643d-427f-461b-a2a6-09ca8377d166 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:48.272928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b60fc-481c-4543-9872-51edebde9d24 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:48.317436 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8ab61-1ac8-4f3c-bb5f-200210413ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:48.318440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8ab61-1ac8-4f3c-bb5f-200210413ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:48.318440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8ab61-1ac8-4f3c-bb5f-200210413ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:48.318904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8ab61-1ac8-4f3c-bb5f-200210413ef1 None admin] PATH_INFO: `/v3/users/a037d04644544ed58c9334d691d41237` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:48.320244 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8ab61-1ac8-4f3c-bb5f-200210413ef1 None admin] RBAC: Authorizing `identity:get_user(user_id=a037d04644544ed58c9334d691d41237)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:48.321455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8ab61-1ac8-4f3c-bb5f-200210413ef1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:48.323504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 147/289] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:48 2022] GET /identity/v3/users/a037d04644544ed58c9334d691d41237 => generated 297 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:48.329096 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8ab61-1ac8-4f3c-bb5f-200210413ef1 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:48.381969 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:48.413891 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:48.422585 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507643d-427f-461b-a2a6-09ca8377d166 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:48.478268 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48508331-77b5-4f15-a3dd-ce9b7731ea06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:48.479340 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48508331-77b5-4f15-a3dd-ce9b7731ea06 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:48.479834 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48508331-77b5-4f15-a3dd-ce9b7731ea06 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:48.480368 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48508331-77b5-4f15-a3dd-ce9b7731ea06 None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:48.482693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48508331-77b5-4f15-a3dd-ce9b7731ea06 None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:48.484553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48508331-77b5-4f15-a3dd-ce9b7731ea06 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:48.494625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 143/290] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:48 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:48.495702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48508331-77b5-4f15-a3dd-ce9b7731ea06 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:48.539672 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8ab61-1ac8-4f3c-bb5f-200210413ef1 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:48.663543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce59d64-5a2e-42b9-b500-5dc8fc907d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:48.664545 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce59d64-5a2e-42b9-b500-5dc8fc907d59 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:48.665047 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce59d64-5a2e-42b9-b500-5dc8fc907d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:48.665581 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce59d64-5a2e-42b9-b500-5dc8fc907d59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:48.762485 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48508331-77b5-4f15-a3dd-ce9b7731ea06 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:48.825565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930ab72-b7ee-4f0d-825a-e41d70829041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:48.827145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930ab72-b7ee-4f0d-825a-e41d70829041 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:48.827938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930ab72-b7ee-4f0d-825a-e41d70829041 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:48.828712 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930ab72-b7ee-4f0d-825a-e41d70829041 None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329/users/7f58e6c51efb409cb64d3c907f8c7998/roles/66ef9c3aa98247eab1c1c726f2f6a5a5` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:48.832829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930ab72-b7ee-4f0d-825a-e41d70829041 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4db61efd90f48a6b63ae8fd15240329, user_id=7f58e6c51efb409cb64d3c907f8c7998, role_id=66ef9c3aa98247eab1c1c726f2f6a5a5)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:48.835841 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930ab72-b7ee-4f0d-825a-e41d70829041 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:48.855375 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce59d64-5a2e-42b9-b500-5dc8fc907d59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:48.856955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce59d64-5a2e-42b9-b500-5dc8fc907d59 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:48.881518 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 144/291] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:29:48 2022] PUT /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329/users/7f58e6c51efb409cb64d3c907f8c7998/roles/66ef9c3aa98247eab1c1c726f2f6a5a5 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:29:48.961958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:49.051846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 148/292] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 10:29:49.057129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930ab72-b7ee-4f0d-825a-e41d70829041 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:49.106710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:49.204846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930ab72-b7ee-4f0d-825a-e41d70829041 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:49.256731 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd60868-9ac8-4808-ac9b-e1d3596fd87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:49.257331 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd60868-9ac8-4808-ac9b-e1d3596fd87f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:49.257649 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd60868-9ac8-4808-ac9b-e1d3596fd87f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:49.257930 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd60868-9ac8-4808-ac9b-e1d3596fd87f None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:49.259257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd60868-9ac8-4808-ac9b-e1d3596fd87f None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:49.260720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd60868-9ac8-4808-ac9b-e1d3596fd87f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:49.262820 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 145/293] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:49 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:49.268885 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce59d64-5a2e-42b9-b500-5dc8fc907d59 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:49.423923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce59d64-5a2e-42b9-b500-5dc8fc907d59 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:49.481883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acac54-5904-4396-99c1-765cc7811346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:49.482921 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acac54-5904-4396-99c1-765cc7811346 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:49.483412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acac54-5904-4396-99c1-765cc7811346 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:49.483895 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acac54-5904-4396-99c1-765cc7811346 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:49.484921 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acac54-5904-4396-99c1-765cc7811346 None admin] RBAC: Adding query filter params (role.id=82be5cfedcb3407f8d14522ef87247ac, scope.project.id=d4db61efd90f48a6b63ae8fd15240329, user.id=a037d04644544ed58c9334d691d41237) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:49.485449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acac54-5904-4396-99c1-765cc7811346 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:49.487157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acac54-5904-4396-99c1-765cc7811346 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:49.494941 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:49.500202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 149/294] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:49 2022] GET /identity/v3/role_assignments?user.id=a037d04644544ed58c9334d691d41237&scope.project.id=d4db61efd90f48a6b63ae8fd15240329&role.id=82be5cfedcb3407f8d14522ef87247ac => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:49.985455 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c3d21-aa24-4e72-8817-35d4dfecbfe4 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:49.986006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c3d21-aa24-4e72-8817-35d4dfecbfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:49.986006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c3d21-aa24-4e72-8817-35d4dfecbfe4 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:49.988344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 146/295] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:49.996178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ba0fb-d4f7-4b06-b3ee-e2b12919ec5e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:49.996671 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ba0fb-d4f7-4b06-b3ee-e2b12919ec5e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:49.997159 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ba0fb-d4f7-4b06-b3ee-e2b12919ec5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:50.009958 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13aef1-2ebe-4952-8b99-055f2ac0a5e9 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:50.010728 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13aef1-2ebe-4952-8b99-055f2ac0a5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:50.011211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13aef1-2ebe-4952-8b99-055f2ac0a5e9 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:50.013968 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 147/296] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:50.019400 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82839b81-92c7-4e39-8ba2-8bcbb147d5fe None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:50.019701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82839b81-92c7-4e39-8ba2-8bcbb147d5fe None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:50.019964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82839b81-92c7-4e39-8ba2-8bcbb147d5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:50.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f40ba0fb-d4f7-4b06-b3ee-e2b12919ec5e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:50.023944 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f40ba0fb-d4f7-4b06-b3ee-e2b12919ec5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:50.043338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-82839b81-92c7-4e39-8ba2-8bcbb147d5fe None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:50.044556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-82839b81-92c7-4e39-8ba2-8bcbb147d5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:50.139991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 150/297] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:50.165542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ba0fb-d4f7-4b06-b3ee-e2b12919ec5e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:50.169449 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 148/298] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:50.199233 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82839b81-92c7-4e39-8ba2-8bcbb147d5fe None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:50.236058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:50.293655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ba0fb-d4f7-4b06-b3ee-e2b12919ec5e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:50.329671 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:50.337429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1e7b1-6952-4a35-b744-c93f835e9f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:50.338150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1e7b1-6952-4a35-b744-c93f835e9f94 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:50.338711 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1e7b1-6952-4a35-b744-c93f835e9f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:50.338959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1e7b1-6952-4a35-b744-c93f835e9f94 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:50.351757 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1e7b1-6952-4a35-b744-c93f835e9f94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-07a1e7b1-6952-4a35-b744-c93f835e9f94 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:50.356867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:50.358133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:50.367387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:50.367387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:50.367387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:50.367387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:50.367387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:50.367387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:50.367387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:50.367387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:50.367387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:50.367387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:50.367387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:50.367387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:50.367387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 151/299] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:29:50.372890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1e7b1-6952-4a35-b744-c93f835e9f94 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:50.381171 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82839b81-92c7-4e39-8ba2-8bcbb147d5fe None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:50.419005 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9177-89a2-4590-b058-3c17ac8e50bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:50.419668 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9177-89a2-4590-b058-3c17ac8e50bf None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:50.419948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9177-89a2-4590-b058-3c17ac8e50bf None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:50.420231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9177-89a2-4590-b058-3c17ac8e50bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:50.421509 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b9177-89a2-4590-b058-3c17ac8e50bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:50.423003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b9177-89a2-4590-b058-3c17ac8e50bf None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:50.424932 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 149/300] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:29:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:50.430942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9177-89a2-4590-b058-3c17ac8e50bf None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:50.529299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:50.541162 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9177-89a2-4590-b058-3c17ac8e50bf None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:50.567835 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1e7b1-6952-4a35-b744-c93f835e9f94 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:50.578818 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a2810-99df-41a3-85f8-50fd7f5d15db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:50.579543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a2810-99df-41a3-85f8-50fd7f5d15db None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:50.579824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a2810-99df-41a3-85f8-50fd7f5d15db None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:50.580105 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a2810-99df-41a3-85f8-50fd7f5d15db None admin] PATH_INFO: `/v3/groups` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:50.580812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a2810-99df-41a3-85f8-50fd7f5d15db None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:50.583066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a2810-99df-41a3-85f8-50fd7f5d15db None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:50.609862 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 150/301] 10.222.0.47 () {60 vars in 1134 bytes} [Wed Aug 3 10:29:50 2022] POST /identity/v3/groups => generated 229 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:29:50.635678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e8f5d-6d3f-41f4-8f73-887466f54fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:50.636729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e8f5d-6d3f-41f4-8f73-887466f54fdd None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:50.637250 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e8f5d-6d3f-41f4-8f73-887466f54fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:50.637798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e8f5d-6d3f-41f4-8f73-887466f54fdd None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:50.638837 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e8f5d-6d3f-41f4-8f73-887466f54fdd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:50.639414 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e8f5d-6d3f-41f4-8f73-887466f54fdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:50.641115 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e8f5d-6d3f-41f4-8f73-887466f54fdd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:50.657165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 152/302] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:50.658997 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a2810-99df-41a3-85f8-50fd7f5d15db None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:50.727678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de3c06-2c0f-4909-84b5-18b372ed8f11 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:50.728357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de3c06-2c0f-4909-84b5-18b372ed8f11 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:50.728852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de3c06-2c0f-4909-84b5-18b372ed8f11 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:50.730551 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 153/303] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:29:50.736452 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e11150-de5e-4d00-99f3-d707bb53b54c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:50.736818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e11150-de5e-4d00-99f3-d707bb53b54c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:50.737068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e11150-de5e-4d00-99f3-d707bb53b54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:50.761787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d6e11150-de5e-4d00-99f3-d707bb53b54c None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:50.763364 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d6e11150-de5e-4d00-99f3-d707bb53b54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:50.809169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a2810-99df-41a3-85f8-50fd7f5d15db None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:50.826522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:50.852321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 154/304] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:50.869613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d1984-9ee3-4cd3-9ab0-4cf18292f4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:50.870501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d1984-9ee3-4cd3-9ab0-4cf18292f4dd None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:50.870944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d1984-9ee3-4cd3-9ab0-4cf18292f4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:50.871374 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d1984-9ee3-4cd3-9ab0-4cf18292f4dd None admin] PATH_INFO: `/v3/users/9fff0f0c6da0406d81936f2fc4ca9cd9` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:50.886313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e11150-de5e-4d00-99f3-d707bb53b54c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:50.887165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d1984-9ee3-4cd3-9ab0-4cf18292f4dd None admin] RBAC: Authorizing `identity:get_user(user_id=9fff0f0c6da0406d81936f2fc4ca9cd9)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:50.888637 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d1984-9ee3-4cd3-9ab0-4cf18292f4dd None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:50.890918 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 151/305] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:50 2022] GET /identity/v3/users/9fff0f0c6da0406d81936f2fc4ca9cd9 => generated 293 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:50.898752 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d1984-9ee3-4cd3-9ab0-4cf18292f4dd None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:50.916704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:51.022146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e11150-de5e-4d00-99f3-d707bb53b54c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:51.025024 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d1984-9ee3-4cd3-9ab0-4cf18292f4dd None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:51.061723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87b6d7-6a29-47c4-8218-ac7aba58f83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:51.061923 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc881c2-e9c9-492d-a5ca-05ce9ca8793f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:51.062310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87b6d7-6a29-47c4-8218-ac7aba58f83c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:51.062413 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc881c2-e9c9-492d-a5ca-05ce9ca8793f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:51.062598 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc881c2-e9c9-492d-a5ca-05ce9ca8793f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:51.062687 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87b6d7-6a29-47c4-8218-ac7aba58f83c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:51.062854 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc881c2-e9c9-492d-a5ca-05ce9ca8793f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:51.062956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87b6d7-6a29-47c4-8218-ac7aba58f83c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:51.063660 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc881c2-e9c9-492d-a5ca-05ce9ca8793f None admin] RBAC: Adding query filter params (role.id=66ef9c3aa98247eab1c1c726f2f6a5a5, user.id=9fff0f0c6da0406d81936f2fc4ca9cd9, scope.system=all) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:51.063964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc881c2-e9c9-492d-a5ca-05ce9ca8793f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:51.065419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc881c2-e9c9-492d-a5ca-05ce9ca8793f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:51.072073 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 152/306] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Aug 3 10:29:50 2022] GET /identity/v3/role_assignments?user.id=9fff0f0c6da0406d81936f2fc4ca9cd9&scope.system=all&role.id=66ef9c3aa98247eab1c1c726f2f6a5a5 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:51.081588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87b6d7-6a29-47c4-8218-ac7aba58f83c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-aa87b6d7-6a29-47c4-8218-ac7aba58f83c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:51.086758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:51.087751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:51.088900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:51.088900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:51.088900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:51.088900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:51.088900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:51.088900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:51.088900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:51.088900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:51.088900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:51.088900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:51.088900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:51.088900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:51.088900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 155/307] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:51.096539 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc881c2-e9c9-492d-a5ca-05ce9ca8793f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:51.185052 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:51.221025 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc881c2-e9c9-492d-a5ca-05ce9ca8793f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:51.261790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b32a8c-9386-4851-b6dc-ec8c8dc7cadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:51.262431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b32a8c-9386-4851-b6dc-ec8c8dc7cadc None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:51.262711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b32a8c-9386-4851-b6dc-ec8c8dc7cadc None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:51.262983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b32a8c-9386-4851-b6dc-ec8c8dc7cadc None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:51.263714 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b32a8c-9386-4851-b6dc-ec8c8dc7cadc None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:51.264017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b32a8c-9386-4851-b6dc-ec8c8dc7cadc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:51.265392 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b32a8c-9386-4851-b6dc-ec8c8dc7cadc None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:51.275491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 153/308] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:51.280838 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87b6d7-6a29-47c4-8218-ac7aba58f83c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:51.343636 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:51.435848 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87b6d7-6a29-47c4-8218-ac7aba58f83c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:51.479936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99baa5fe-0111-4527-a101-d823b57be77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:51.480675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99baa5fe-0111-4527-a101-d823b57be77b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:51.480961 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99baa5fe-0111-4527-a101-d823b57be77b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:51.481240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99baa5fe-0111-4527-a101-d823b57be77b None admin] PATH_INFO: `/v3/users/f8ba9a843c0442ca81a2cf3a524e5fa3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:51.482783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99baa5fe-0111-4527-a101-d823b57be77b None admin] RBAC: Authorizing `identity:get_user(user_id=f8ba9a843c0442ca81a2cf3a524e5fa3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:51.484858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99baa5fe-0111-4527-a101-d823b57be77b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:51.487155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 156/309] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:51 2022] GET /identity/v3/users/f8ba9a843c0442ca81a2cf3a524e5fa3 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:51.493052 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b32a8c-9386-4851-b6dc-ec8c8dc7cadc None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:51.677046 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f454c-0ed6-4ddf-83f2-6b365a974b57 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:51.677607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f454c-0ed6-4ddf-83f2-6b365a974b57 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:51.677607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f454c-0ed6-4ddf-83f2-6b365a974b57 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:51.678951 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 157/310] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:51.682996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b32a8c-9386-4851-b6dc-ec8c8dc7cadc None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:51.687813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a26c9-a0f2-4045-b337-d49041505f79 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:51.688233 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a26c9-a0f2-4045-b337-d49041505f79 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:51.688769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a26c9-a0f2-4045-b337-d49041505f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:51.710006 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f88a26c9-a0f2-4045-b337-d49041505f79 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:51.711338 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f88a26c9-a0f2-4045-b337-d49041505f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:51.739691 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd50c8-d179-4516-b3de-cfb797eef530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:51.741494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd50c8-d179-4516-b3de-cfb797eef530 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:51.742225 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd50c8-d179-4516-b3de-cfb797eef530 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:51.743068 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd50c8-d179-4516-b3de-cfb797eef530 None admin] PATH_INFO: `/v3/system/users/f8ba9a843c0442ca81a2cf3a524e5fa3/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:51.745977 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd50c8-d179-4516-b3de-cfb797eef530 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f8ba9a843c0442ca81a2cf3a524e5fa3, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:51.747501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd50c8-d179-4516-b3de-cfb797eef530 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:51.749302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:51.765518 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 154/311] 10.222.0.47 () {58 vars in 1334 bytes} [Wed Aug 3 10:29:51 2022] PUT /identity/v3/system/users/f8ba9a843c0442ca81a2cf3a524e5fa3/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:29:51.804388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 158/312] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:51.828441 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd50c8-d179-4516-b3de-cfb797eef530 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:51.963544 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd50c8-d179-4516-b3de-cfb797eef530 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:52.005572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50585b-b009-48f6-bcac-134fa2faade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:52.006252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50585b-b009-48f6-bcac-134fa2faade2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:52.006534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50585b-b009-48f6-bcac-134fa2faade2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:52.006814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50585b-b009-48f6-bcac-134fa2faade2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:52.018436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50585b-b009-48f6-bcac-134fa2faade2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-4a50585b-b009-48f6-bcac-134fa2faade2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:52.027864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:52.029737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:52.030771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:52.030771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:52.030771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:52.030771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:52.030771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:52.030771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:52.030771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:52.030771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:52.030771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:52.030771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:52.030771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:52.030771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:52.030771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 155/313] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:52.036198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a26c9-a0f2-4045-b337-d49041505f79 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:52.097319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:52.170071 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a26c9-a0f2-4045-b337-d49041505f79 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:52.207956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079c9de-6285-460d-88ba-3151e92c0e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:52.208976 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079c9de-6285-460d-88ba-3151e92c0e95 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:52.209455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079c9de-6285-460d-88ba-3151e92c0e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:52.209977 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079c9de-6285-460d-88ba-3151e92c0e95 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:52.210911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079c9de-6285-460d-88ba-3151e92c0e95 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:52.211417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079c9de-6285-460d-88ba-3151e92c0e95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:52.213929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079c9de-6285-460d-88ba-3151e92c0e95 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:52.228546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 159/314] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:52.236631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50585b-b009-48f6-bcac-134fa2faade2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:52.303067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:52.363334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50585b-b009-48f6-bcac-134fa2faade2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:52.405058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6011eb-e578-4c4b-9d28-1e5150fedaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:52.406115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6011eb-e578-4c4b-9d28-1e5150fedaef None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:52.406627 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6011eb-e578-4c4b-9d28-1e5150fedaef None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:52.407133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6011eb-e578-4c4b-9d28-1e5150fedaef None admin] PATH_INFO: `/v3/users/7f58e6c51efb409cb64d3c907f8c7998` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:52.408604 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6011eb-e578-4c4b-9d28-1e5150fedaef None admin] RBAC: Authorizing `identity:get_user(user_id=7f58e6c51efb409cb64d3c907f8c7998)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:52.410745 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6011eb-e578-4c4b-9d28-1e5150fedaef None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:52.412997 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 156/315] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:52 2022] GET /identity/v3/users/7f58e6c51efb409cb64d3c907f8c7998 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:52.418832 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079c9de-6285-460d-88ba-3151e92c0e95 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:52.594754 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079c9de-6285-460d-88ba-3151e92c0e95 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:52.597984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:52.634379 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f662b-7df1-4bd0-8316-d1dfe72f3091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:52.635015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f662b-7df1-4bd0-8316-d1dfe72f3091 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:52.635299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f662b-7df1-4bd0-8316-d1dfe72f3091 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:52.635577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f662b-7df1-4bd0-8316-d1dfe72f3091 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:52.710589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f662b-7df1-4bd0-8316-d1dfe72f3091 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:52.712067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f662b-7df1-4bd0-8316-d1dfe72f3091 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:52.824708 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 160/316] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:29:52.830917 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6011eb-e578-4c4b-9d28-1e5150fedaef None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:52.849080 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbe4cb-0037-4d63-9ffe-09d41b0649b1 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:52.849370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbe4cb-0037-4d63-9ffe-09d41b0649b1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:52.849610 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbe4cb-0037-4d63-9ffe-09d41b0649b1 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:52.851288 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 161/317] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:52.856955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff442c1-da2d-4e17-a79a-7587a3987753 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:52.857283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff442c1-da2d-4e17-a79a-7587a3987753 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:52.857585 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff442c1-da2d-4e17-a79a-7587a3987753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:52.883397 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-fff442c1-da2d-4e17-a79a-7587a3987753 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:52.885985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:52.891001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fff442c1-da2d-4e17-a79a-7587a3987753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:52.985129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6011eb-e578-4c4b-9d28-1e5150fedaef None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:53.013078 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 162/318] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:53.025947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:53.038414 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff442c1-da2d-4e17-a79a-7587a3987753 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:53.040177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413bd94-4201-4a0f-a653-3d7d2f46c455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:53.041703 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413bd94-4201-4a0f-a653-3d7d2f46c455 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:53.043432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413bd94-4201-4a0f-a653-3d7d2f46c455 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:53.044017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413bd94-4201-4a0f-a653-3d7d2f46c455 None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:53.046444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413bd94-4201-4a0f-a653-3d7d2f46c455 None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:53.049283 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413bd94-4201-4a0f-a653-3d7d2f46c455 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:53.054808 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 157/319] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:52 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:53.060721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413bd94-4201-4a0f-a653-3d7d2f46c455 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:53.224002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff442c1-da2d-4e17-a79a-7587a3987753 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:53.225314 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413bd94-4201-4a0f-a653-3d7d2f46c455 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:53.264857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c793879-02ab-46b5-99eb-5fcebc735fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:53.265681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c793879-02ab-46b5-99eb-5fcebc735fca None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:53.266072 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c793879-02ab-46b5-99eb-5fcebc735fca None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:53.266481 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c793879-02ab-46b5-99eb-5fcebc735fca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:53.267473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c793879-02ab-46b5-99eb-5fcebc735fca None admin] RBAC: Adding query filter params (role.id=66ef9c3aa98247eab1c1c726f2f6a5a5, scope.project.id=d4db61efd90f48a6b63ae8fd15240329, user.id=7f58e6c51efb409cb64d3c907f8c7998) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:53.267957 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c793879-02ab-46b5-99eb-5fcebc735fca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:53.270097 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c793879-02ab-46b5-99eb-5fcebc735fca None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:53.277369 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 158/320] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:29:53 2022] GET /identity/v3/role_assignments?user.id=7f58e6c51efb409cb64d3c907f8c7998&scope.project.id=d4db61efd90f48a6b63ae8fd15240329&role.id=66ef9c3aa98247eab1c1c726f2f6a5a5 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:53.280007 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:53.297795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c06db-9843-4bf6-afef-4ea320ee27ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:53.298727 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c06db-9843-4bf6-afef-4ea320ee27ea None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:53.299030 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c06db-9843-4bf6-afef-4ea320ee27ea None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:53.299327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c06db-9843-4bf6-afef-4ea320ee27ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:53.301891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c06db-9843-4bf6-afef-4ea320ee27ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:53.304325 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c06db-9843-4bf6-afef-4ea320ee27ea None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:53.307276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 163/321] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:29:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:53.316993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c793879-02ab-46b5-99eb-5fcebc735fca None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:53.435660 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:53.484487 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c793879-02ab-46b5-99eb-5fcebc735fca None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:53.493994 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ee863-50b1-4967-8dda-928d96274eb6 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:53.494612 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ee863-50b1-4967-8dda-928d96274eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:53.495112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ee863-50b1-4967-8dda-928d96274eb6 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:53.497138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 164/322] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:53.503001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ebe79-db91-4ec6-af0a-b427c4b46e15 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:53.504163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ebe79-db91-4ec6-af0a-b427c4b46e15 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:53.504659 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ebe79-db91-4ec6-af0a-b427c4b46e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:53.527996 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c72ebe79-db91-4ec6-af0a-b427c4b46e15 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:53.529918 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c72ebe79-db91-4ec6-af0a-b427c4b46e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:53.530960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7491a6-fd11-426a-8c1a-21bdc52c4f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:53.531793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7491a6-fd11-426a-8c1a-21bdc52c4f81 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:53.532088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7491a6-fd11-426a-8c1a-21bdc52c4f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:53.532457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7491a6-fd11-426a-8c1a-21bdc52c4f81 None admin] PATH_INFO: `/v3/groups` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:53.533844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7491a6-fd11-426a-8c1a-21bdc52c4f81 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:53.536064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7491a6-fd11-426a-8c1a-21bdc52c4f81 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:53.557543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 159/323] 10.222.0.47 () {60 vars in 1134 bytes} [Wed Aug 3 10:29:53 2022] POST /identity/v3/groups => generated 226 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:29:53.653120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 165/324] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:53.676654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7491a6-fd11-426a-8c1a-21bdc52c4f81 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:53.845086 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7491a6-fd11-426a-8c1a-21bdc52c4f81 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:53.872790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:53.899517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b9a00-2a53-4bcd-8c40-735194fb6383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:53.900593 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b9a00-2a53-4bcd-8c40-735194fb6383 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:53.901079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b9a00-2a53-4bcd-8c40-735194fb6383 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:53.901603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b9a00-2a53-4bcd-8c40-735194fb6383 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:53.920253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b9a00-2a53-4bcd-8c40-735194fb6383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-540b9a00-2a53-4bcd-8c40-735194fb6383 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:53.926824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:53.928996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:53.930880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:53.930880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:53.930880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:53.930880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:53.930880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:53.930880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:53.930880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:53.930880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:53.930880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:53.930880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:53.930880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:53.930880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:53.930880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 160/325] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:29:53.936998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ebe79-db91-4ec6-af0a-b427c4b46e15 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:54.102469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ebe79-db91-4ec6-af0a-b427c4b46e15 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:54.135823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:54.145258 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b995d-a932-4c92-8cbd-95638d16a159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:54.145979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b995d-a932-4c92-8cbd-95638d16a159 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:54.146259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b995d-a932-4c92-8cbd-95638d16a159 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:54.146532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b995d-a932-4c92-8cbd-95638d16a159 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:54.147426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b995d-a932-4c92-8cbd-95638d16a159 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:54.147732 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b995d-a932-4c92-8cbd-95638d16a159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:54.149199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b995d-a932-4c92-8cbd-95638d16a159 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:54.159193 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 166/326] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:54.167811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b9a00-2a53-4bcd-8c40-735194fb6383 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:54.187065 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec726d3d-9082-4c66-8b67-242e133b1af2 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:54.187395 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec726d3d-9082-4c66-8b67-242e133b1af2 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:54.187607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec726d3d-9082-4c66-8b67-242e133b1af2 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:54.189396 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 167/327] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:29:54.196199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eaeedd-a99b-4700-b64c-525eb82ccc11 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:54.197109 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eaeedd-a99b-4700-b64c-525eb82ccc11 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:54.198010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eaeedd-a99b-4700-b64c-525eb82ccc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:54.243028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-44eaeedd-a99b-4700-b64c-525eb82ccc11 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:54.245626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-44eaeedd-a99b-4700-b64c-525eb82ccc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:54.409789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b9a00-2a53-4bcd-8c40-735194fb6383 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:54.411910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 168/328] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:54.441540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eaeedd-a99b-4700-b64c-525eb82ccc11 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:54.492717 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef705bba-f54b-417c-a2e4-f8000e7938dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:54.493396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef705bba-f54b-417c-a2e4-f8000e7938dd None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:54.493713 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef705bba-f54b-417c-a2e4-f8000e7938dd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:54.495363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef705bba-f54b-417c-a2e4-f8000e7938dd None admin] PATH_INFO: `/v3/users/9fff0f0c6da0406d81936f2fc4ca9cd9` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:54.496871 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef705bba-f54b-417c-a2e4-f8000e7938dd None admin] RBAC: Authorizing `identity:get_user(user_id=9fff0f0c6da0406d81936f2fc4ca9cd9)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:54.498496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef705bba-f54b-417c-a2e4-f8000e7938dd None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:54.500871 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 161/329] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:54 2022] GET /identity/v3/users/9fff0f0c6da0406d81936f2fc4ca9cd9 => generated 293 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:54.508837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef705bba-f54b-417c-a2e4-f8000e7938dd None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:54.600122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eaeedd-a99b-4700-b64c-525eb82ccc11 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:54.623813 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:54.640540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:54.672119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f9dde-73c6-49ad-929f-b6a148b32f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:54.672717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f9dde-73c6-49ad-929f-b6a148b32f85 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:54.673007 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f9dde-73c6-49ad-929f-b6a148b32f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:54.673288 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f9dde-73c6-49ad-929f-b6a148b32f85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:54.702737 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f9dde-73c6-49ad-929f-b6a148b32f85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-6d8f9dde-73c6-49ad-929f-b6a148b32f85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:54.708502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:54.710296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:54.711882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:54.711882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:54.711882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:54.711882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:54.711882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:54.711882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:54.711882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:54.711882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:54.711882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:54.711882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:54.711882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:54.711882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:54.718263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 169/330] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:54.725760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f9dde-73c6-49ad-929f-b6a148b32f85 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:54.797029 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef705bba-f54b-417c-a2e4-f8000e7938dd None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:54.859705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f9dde-73c6-49ad-929f-b6a148b32f85 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:54.892671 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fa5a8-a7e5-4169-9d3c-ee92ffc02aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:54.894293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fa5a8-a7e5-4169-9d3c-ee92ffc02aff None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:54.895028 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fa5a8-a7e5-4169-9d3c-ee92ffc02aff None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:54.895829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fa5a8-a7e5-4169-9d3c-ee92ffc02aff None admin] PATH_INFO: `/v3/system/users/9fff0f0c6da0406d81936f2fc4ca9cd9/roles/66ef9c3aa98247eab1c1c726f2f6a5a5` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:54.899103 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fa5a8-a7e5-4169-9d3c-ee92ffc02aff None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9fff0f0c6da0406d81936f2fc4ca9cd9, role_id=66ef9c3aa98247eab1c1c726f2f6a5a5)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:54.902536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fa5a8-a7e5-4169-9d3c-ee92ffc02aff None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:54.917240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de017769-04c9-4b3d-bab4-0d6516a7c93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:54.917240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de017769-04c9-4b3d-bab4-0d6516a7c93c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:54.917240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de017769-04c9-4b3d-bab4-0d6516a7c93c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:54.917240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de017769-04c9-4b3d-bab4-0d6516a7c93c None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:54.917929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de017769-04c9-4b3d-bab4-0d6516a7c93c None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:54.917929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de017769-04c9-4b3d-bab4-0d6516a7c93c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:54.918432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de017769-04c9-4b3d-bab4-0d6516a7c93c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:54.927683 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 162/331] 10.222.0.47 () {58 vars in 1334 bytes} [Wed Aug 3 10:29:54 2022] PUT /identity/v3/system/users/9fff0f0c6da0406d81936f2fc4ca9cd9/roles/66ef9c3aa98247eab1c1c726f2f6a5a5 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:29:54.938809 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 170/332] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:29:54.942124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fa5a8-a7e5-4169-9d3c-ee92ffc02aff None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:55.185692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fa5a8-a7e5-4169-9d3c-ee92ffc02aff None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:55.210050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:55.258844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a74055-912b-425d-bfa9-78fe7d4dab3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:55.260382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a74055-912b-425d-bfa9-78fe7d4dab3f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:55.261116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a74055-912b-425d-bfa9-78fe7d4dab3f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:55.261921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a74055-912b-425d-bfa9-78fe7d4dab3f None admin] PATH_INFO: `/v3/users/f8ba9a843c0442ca81a2cf3a524e5fa3` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:55.264673 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a74055-912b-425d-bfa9-78fe7d4dab3f None admin] RBAC: Authorizing `identity:get_user(user_id=f8ba9a843c0442ca81a2cf3a524e5fa3)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:55.267977 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a74055-912b-425d-bfa9-78fe7d4dab3f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:55.272363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 163/333] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:29:54 2022] GET /identity/v3/users/f8ba9a843c0442ca81a2cf3a524e5fa3 => generated 293 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:55.278071 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de017769-04c9-4b3d-bab4-0d6516a7c93c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:55.344244 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:55.451410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de017769-04c9-4b3d-bab4-0d6516a7c93c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:55.535779 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a58c4e-d175-4ec9-86ce-77d292f5df65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:55.537225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a58c4e-d175-4ec9-86ce-77d292f5df65 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:55.537976 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a58c4e-d175-4ec9-86ce-77d292f5df65 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:55.538702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a58c4e-d175-4ec9-86ce-77d292f5df65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:55.540504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a58c4e-d175-4ec9-86ce-77d292f5df65 None admin] RBAC: Adding query filter params (role.id=82be5cfedcb3407f8d14522ef87247ac, user.id=f8ba9a843c0442ca81a2cf3a524e5fa3, scope.system=all) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:55.541412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a58c4e-d175-4ec9-86ce-77d292f5df65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:55.544407 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a58c4e-d175-4ec9-86ce-77d292f5df65 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:55.572936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 171/334] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Aug 3 10:29:55 2022] GET /identity/v3/role_assignments?user.id=f8ba9a843c0442ca81a2cf3a524e5fa3&scope.system=all&role.id=82be5cfedcb3407f8d14522ef87247ac => generated 510 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:57.245139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817d40e-eaff-4214-8545-ba9afc28f5c4 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:57.246594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817d40e-eaff-4214-8545-ba9afc28f5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:57.247312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817d40e-eaff-4214-8545-ba9afc28f5c4 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:57.250430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 164/335] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:57.259468 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b6246-18ea-49d3-83a3-875393a216fa None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:57.260737 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b6246-18ea-49d3-83a3-875393a216fa None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:57.261397 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b6246-18ea-49d3-83a3-875393a216fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:57.299238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d80b6246-18ea-49d3-83a3-875393a216fa None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:57.302400 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d80b6246-18ea-49d3-83a3-875393a216fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:57.453382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 172/336] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:57.517839 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817d40e-eaff-4214-8545-ba9afc28f5c4 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:57.772463 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817d40e-eaff-4214-8545-ba9afc28f5c4 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:57.858962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429cd17-865c-4552-99d2-e83ee7a5ac9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:57.860699 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429cd17-865c-4552-99d2-e83ee7a5ac9c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:57.861599 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429cd17-865c-4552-99d2-e83ee7a5ac9c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:57.862934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429cd17-865c-4552-99d2-e83ee7a5ac9c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:57.899294 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2429cd17-865c-4552-99d2-e83ee7a5ac9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-2429cd17-865c-4552-99d2-e83ee7a5ac9c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:57.911035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:57.912741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:57.914376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:57.914376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:57.914376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:57.914376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:57.914376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:57.914376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:57.914376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:57.914376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:57.914376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:57.914376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:57.914376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:29:57.914376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:57.919286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 165/337] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:29:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:57.929281 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b6246-18ea-49d3-83a3-875393a216fa None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:58.002028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:58.115937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b6246-18ea-49d3-83a3-875393a216fa None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:58.166982 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97569c-79a4-40c8-bd02-e8ef885919ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:58.167629 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97569c-79a4-40c8-bd02-e8ef885919ca None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:58.167915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97569c-79a4-40c8-bd02-e8ef885919ca None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:58.168197 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97569c-79a4-40c8-bd02-e8ef885919ca None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:58.168954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97569c-79a4-40c8-bd02-e8ef885919ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:58.169265 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97569c-79a4-40c8-bd02-e8ef885919ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:58.170724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97569c-79a4-40c8-bd02-e8ef885919ca None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:58.184821 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 173/338] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:29:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:58.192718 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429cd17-865c-4552-99d2-e83ee7a5ac9c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:58.249154 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ce411-723e-4bb6-9e80-eedcf012f661 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:58.249514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ce411-723e-4bb6-9e80-eedcf012f661 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:58.249812 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ce411-723e-4bb6-9e80-eedcf012f661 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:58.251626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 174/339] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:58.264222 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c5ba0-432b-4863-b674-86d6ff5a74cf None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:58.264856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c5ba0-432b-4863-b674-86d6ff5a74cf None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:58.264856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c5ba0-432b-4863-b674-86d6ff5a74cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:58.292494 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ee5c5ba0-432b-4863-b674-86d6ff5a74cf None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:58.293924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ee5c5ba0-432b-4863-b674-86d6ff5a74cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:58.308741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:58.421016 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 175/340] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:58.422591 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43eb44b-ddf5-422e-9e2e-0694dd804613 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:58.422885 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43eb44b-ddf5-422e-9e2e-0694dd804613 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:58.423165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43eb44b-ddf5-422e-9e2e-0694dd804613 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:58.424684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 176/341] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:58.446788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9e565-542a-43fe-8274-b7253665c6f0 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:58.447089 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9e565-542a-43fe-8274-b7253665c6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:58.447354 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9e565-542a-43fe-8274-b7253665c6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:58.454090 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429cd17-865c-4552-99d2-e83ee7a5ac9c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:58.478616 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ffa9e565-542a-43fe-8274-b7253665c6f0 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:58.479860 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ffa9e565-542a-43fe-8274-b7253665c6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:58.525858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:58.544281 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833fda3-b4b8-410d-99ee-d9b0d19b573b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:58.544934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833fda3-b4b8-410d-99ee-d9b0d19b573b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:58.545221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833fda3-b4b8-410d-99ee-d9b0d19b573b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:58.545520 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833fda3-b4b8-410d-99ee-d9b0d19b573b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:58.633651 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 177/342] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:29:58.634931 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9e565-542a-43fe-8274-b7253665c6f0 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:58.671442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833fda3-b4b8-410d-99ee-d9b0d19b573b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:58.673211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833fda3-b4b8-410d-99ee-d9b0d19b573b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:58.740200 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:58.815225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9e565-542a-43fe-8274-b7253665c6f0 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:58.833955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 166/343] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:29:58.840651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb338f2e-f382-453e-bbb0-6338b01adaed None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:58.841514 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb338f2e-f382-453e-bbb0-6338b01adaed None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:58.841996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb338f2e-f382-453e-bbb0-6338b01adaed None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:58.843851 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 167/344] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:58.844963 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb338f2e-f382-453e-bbb0-6338b01adaed None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:58.866955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bf384-986e-423d-8fcc-1cb63eb20926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:58.867631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bf384-986e-423d-8fcc-1cb63eb20926 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:58.867909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bf384-986e-423d-8fcc-1cb63eb20926 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:58.868178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bf384-986e-423d-8fcc-1cb63eb20926 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:58.882103 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bf384-986e-423d-8fcc-1cb63eb20926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-243bf384-986e-423d-8fcc-1cb63eb20926 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:58.888091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:58.889049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:58.889882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:58.889882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:58.889882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:58.889882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:58.889882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:58.889882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:58.889882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:58.889882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:58.889882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:58.889882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:58.889882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:58.889882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:58.894311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 178/345] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:58.895457 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bf384-986e-423d-8fcc-1cb63eb20926 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:58.915724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:58.998211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb338f2e-f382-453e-bbb0-6338b01adaed None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:59.041413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bf384-986e-423d-8fcc-1cb63eb20926 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:59.053122 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ea79e-ae94-4c41-92bb-ac0c1dde4081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:59.055165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ea79e-ae94-4c41-92bb-ac0c1dde4081 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:59.056515 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ea79e-ae94-4c41-92bb-ac0c1dde4081 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:59.057781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ea79e-ae94-4c41-92bb-ac0c1dde4081 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:59.074531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:59.083411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ea79e-ae94-4c41-92bb-ac0c1dde4081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:59.086751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf83090-55d1-47f3-91f1-4a94af3c1eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:59.087419 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf83090-55d1-47f3-91f1-4a94af3c1eeb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:59.087717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf83090-55d1-47f3-91f1-4a94af3c1eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:59.088094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf83090-55d1-47f3-91f1-4a94af3c1eeb None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:59.089675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf83090-55d1-47f3-91f1-4a94af3c1eeb None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:59.091138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf83090-55d1-47f3-91f1-4a94af3c1eeb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:59.092850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 179/346] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:29:58 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:59.096056 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed094344-0747-4d5f-81a4-2e411a3fbd07 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:59.096341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed094344-0747-4d5f-81a4-2e411a3fbd07 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:59.096602 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed094344-0747-4d5f-81a4-2e411a3fbd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-379ea79e-ae94-4c41-92bb-ac0c1dde4081 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:59.098971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:59.100592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:59.102296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:59.102296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:59.102296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:59.102296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:59.102296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:59.102296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:59.102296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:59.102296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:59.102296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:59.102296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:59.102296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:29:59.102296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:29:59.106737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 168/347] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:59.108602 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ea79e-ae94-4c41-92bb-ac0c1dde4081 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:59.118188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ed094344-0747-4d5f-81a4-2e411a3fbd07 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:59.119392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ed094344-0747-4d5f-81a4-2e411a3fbd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:59.197512 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 180/348] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:59.198458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed094344-0747-4d5f-81a4-2e411a3fbd07 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:59.219111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:59.297765 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ea79e-ae94-4c41-92bb-ac0c1dde4081 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:59.341232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae7177-d422-4d8b-9fda-3533699b1683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:59.341232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae7177-d422-4d8b-9fda-3533699b1683 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:59.341232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae7177-d422-4d8b-9fda-3533699b1683 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:59.342321 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae7177-d422-4d8b-9fda-3533699b1683 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:59.342321 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae7177-d422-4d8b-9fda-3533699b1683 None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:59.342849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae7177-d422-4d8b-9fda-3533699b1683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:59.345457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae7177-d422-4d8b-9fda-3533699b1683 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:59.354051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 169/349] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:59.354949 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae7177-d422-4d8b-9fda-3533699b1683 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:59.368877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed094344-0747-4d5f-81a4-2e411a3fbd07 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:59.416837 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7fe50-2a56-4366-ae86-61c3d6206227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:59.417576 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7fe50-2a56-4366-ae86-61c3d6206227 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:59.417849 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7fe50-2a56-4366-ae86-61c3d6206227 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:59.418092 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7fe50-2a56-4366-ae86-61c3d6206227 None admin] PATH_INFO: `/v3/groups/ebeaaf50bf9441d7992937759eff8370` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:59.426533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7fe50-2a56-4366-ae86-61c3d6206227 None admin] RBAC: Authorizing `identity:get_group(group_id=ebeaaf50bf9441d7992937759eff8370)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:59.427999 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7fe50-2a56-4366-ae86-61c3d6206227 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:59.430001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 181/350] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 3 10:29:59 2022] GET /identity/v3/groups/ebeaaf50bf9441d7992937759eff8370 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:59.431141 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174b7a8-d3d8-485c-9c72-3b999a619d87 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:59.431422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174b7a8-d3d8-485c-9c72-3b999a619d87 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:59.431699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174b7a8-d3d8-485c-9c72-3b999a619d87 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:59.433299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 182/351] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:59.438093 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174b7a8-d3d8-485c-9c72-3b999a619d87 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:59.462199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:59.528679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae7177-d422-4d8b-9fda-3533699b1683 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:59.564979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:59.582546 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401958a-256d-44fd-8164-3bc53ae6c178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:59.583823 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401958a-256d-44fd-8164-3bc53ae6c178 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:59.584273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401958a-256d-44fd-8164-3bc53ae6c178 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:59.584697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401958a-256d-44fd-8164-3bc53ae6c178 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:59.586030 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401958a-256d-44fd-8164-3bc53ae6c178 None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:29:59.589645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401958a-256d-44fd-8164-3bc53ae6c178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:59.589645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401958a-256d-44fd-8164-3bc53ae6c178 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:29:59.604501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 170/352] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:29:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:29:59.606017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5253612-c8f2-41e3-83da-6871267a1803 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:59.606810 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5253612-c8f2-41e3-83da-6871267a1803 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:59.607608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5253612-c8f2-41e3-83da-6871267a1803 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:59.609334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 171/353] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:29:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:29:59.613385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5253612-c8f2-41e3-83da-6871267a1803 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:59.616164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174b7a8-d3d8-485c-9c72-3b999a619d87 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:59.670116 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05742e-84c1-4109-9b0b-0661be8b4135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:59.671366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05742e-84c1-4109-9b0b-0661be8b4135 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:59.671935 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05742e-84c1-4109-9b0b-0661be8b4135 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:59.672327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05742e-84c1-4109-9b0b-0661be8b4135 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:59.697362 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05742e-84c1-4109-9b0b-0661be8b4135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-1a05742e-84c1-4109-9b0b-0661be8b4135 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:29:59.705560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:29:59.708000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:29:59.709880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:29:59.709880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:29:59.709880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:29:59.709880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:29:59.709880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:29:59.709880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:29:59.709880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:29:59.709880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:29:59.709880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:29:59.709880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:29:59.709880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 10:29:59.709880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:29:59.713749 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 183/354] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:29:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:29:59.713749 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6bd84-868e-459b-9b5f-7b57e1f13cd2 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:59.713749 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6bd84-868e-459b-9b5f-7b57e1f13cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:59.714646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6bd84-868e-459b-9b5f-7b57e1f13cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:59.740238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c1a6bd84-868e-459b-9b5f-7b57e1f13cd2 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:29:59.741782 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c1a6bd84-868e-459b-9b5f-7b57e1f13cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:29:59.800941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5253612-c8f2-41e3-83da-6871267a1803 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:29:59.837955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:59.855847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 184/355] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:29:59.857267 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6bd84-868e-459b-9b5f-7b57e1f13cd2 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:29:59.858340 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a3405-06aa-4a35-908a-7b418944889b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:29:59.859218 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a3405-06aa-4a35-908a-7b418944889b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:29:59.859657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a3405-06aa-4a35-908a-7b418944889b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:29:59.860057 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a3405-06aa-4a35-908a-7b418944889b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:29:59.976768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:29:59.986078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a3405-06aa-4a35-908a-7b418944889b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:29:59.988199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a3405-06aa-4a35-908a-7b418944889b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:00.033240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6bd84-868e-459b-9b5f-7b57e1f13cd2 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:00.090434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac3f27-dd42-4149-8cb4-8f69f4638d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:00.091237 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac3f27-dd42-4149-8cb4-8f69f4638d83 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:00.091715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac3f27-dd42-4149-8cb4-8f69f4638d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:00.092098 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac3f27-dd42-4149-8cb4-8f69f4638d83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:00.093120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac3f27-dd42-4149-8cb4-8f69f4638d83 None admin] RBAC: Adding query filter params (group.id=ebeaaf50bf9441d7992937759eff8370, role.id=cde4978cbb9648639ff720b5ca82886b, scope.project.id=998a52ff5527415da541925c79fddaab) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:00.093604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac3f27-dd42-4149-8cb4-8f69f4638d83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:00.095490 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac3f27-dd42-4149-8cb4-8f69f4638d83 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:00.103545 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 185/356] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 3 10:29:59 2022] GET /identity/v3/role_assignments?group.id=ebeaaf50bf9441d7992937759eff8370&scope.project.id=998a52ff5527415da541925c79fddaab&role.id=cde4978cbb9648639ff720b5ca82886b => generated 260 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:00.105348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac3f27-dd42-4149-8cb4-8f69f4638d83 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:00.179288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 172/357] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:29:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:00.182082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706ac87-46a0-4303-9659-80aef6e603cf None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:00.182630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706ac87-46a0-4303-9659-80aef6e603cf None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:00.183154 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706ac87-46a0-4303-9659-80aef6e603cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:00.218629 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-1706ac87-46a0-4303-9659-80aef6e603cf None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:00.220939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1706ac87-46a0-4303-9659-80aef6e603cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:00.256523 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:00.305074 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac3f27-dd42-4149-8cb4-8f69f4638d83 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:00.346514 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 173/358] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:00.348560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706ac87-46a0-4303-9659-80aef6e603cf None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:00.364351 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:00.378028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2ac17-d748-449d-9814-2ccb79fad1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:00.379029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ac17-d748-449d-9814-2ccb79fad1cf None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:00.379508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ac17-d748-449d-9814-2ccb79fad1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:00.379962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ac17-d748-449d-9814-2ccb79fad1cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:00.495897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2ac17-d748-449d-9814-2ccb79fad1cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:00.498971 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2ac17-d748-449d-9814-2ccb79fad1cf None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:00.528463 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706ac87-46a0-4303-9659-80aef6e603cf None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:00.573945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf527f5-28e3-4669-9f55-e88ea58613f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:00.574986 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf527f5-28e3-4669-9f55-e88ea58613f9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:00.575543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf527f5-28e3-4669-9f55-e88ea58613f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:00.576067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf527f5-28e3-4669-9f55-e88ea58613f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:00.577049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf527f5-28e3-4669-9f55-e88ea58613f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:00.577384 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf527f5-28e3-4669-9f55-e88ea58613f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:00.578791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf527f5-28e3-4669-9f55-e88ea58613f9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:00.587752 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 174/359] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:30:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:00.588956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf527f5-28e3-4669-9f55-e88ea58613f9 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:00.614830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:00.663891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 186/360] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:00.666218 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2ac17-d748-449d-9814-2ccb79fad1cf None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:00.719126 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:00.727160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf527f5-28e3-4669-9f55-e88ea58613f9 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:00.787156 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221acfc-2cbc-405b-810b-6cdf316abff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:00.788602 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221acfc-2cbc-405b-810b-6cdf316abff3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:00.788795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221acfc-2cbc-405b-810b-6cdf316abff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:00.789273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221acfc-2cbc-405b-810b-6cdf316abff3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:00.808069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2221acfc-2cbc-405b-810b-6cdf316abff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-2221acfc-2cbc-405b-810b-6cdf316abff3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:00.814284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:00.816062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:00.817760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:00.817760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:00.817760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:00.817760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:00.817760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:00.817760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:00.817760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:00.817760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:00.817760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:00.817760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:00.817760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:30:00.817760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:00.821069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 175/361] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:30:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:00.823161 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221acfc-2cbc-405b-810b-6cdf316abff3 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:00.836104 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2ac17-d748-449d-9814-2ccb79fad1cf None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:00.886349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5923f-044c-46c2-af1d-bc7f7ac3bac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:00.887014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5923f-044c-46c2-af1d-bc7f7ac3bac2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:00.887301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5923f-044c-46c2-af1d-bc7f7ac3bac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:00.887571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5923f-044c-46c2-af1d-bc7f7ac3bac2 None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:00.888876 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5923f-044c-46c2-af1d-bc7f7ac3bac2 None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:00.890555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5923f-044c-46c2-af1d-bc7f7ac3bac2 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:00.892729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 187/362] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:30:00 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:00.894158 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5923f-044c-46c2-af1d-bc7f7ac3bac2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:00.936945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:00.966879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221acfc-2cbc-405b-810b-6cdf316abff3 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:01.012393 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93cfb2-fc4f-4466-8eac-4f6988938dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:01.013518 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93cfb2-fc4f-4466-8eac-4f6988938dee None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:01.013518 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93cfb2-fc4f-4466-8eac-4f6988938dee None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:01.013704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93cfb2-fc4f-4466-8eac-4f6988938dee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:01.029054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93cfb2-fc4f-4466-8eac-4f6988938dee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-5a93cfb2-fc4f-4466-8eac-4f6988938dee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:01.036036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:01.038301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:01.040141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:01.040141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:01.040141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:01.040141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:01.040141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:01.040141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:01.040141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:01.040141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:01.040141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:01.040141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:01.040141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:30:01.040141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:01.040141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 176/363] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:30:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:01.040141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93cfb2-fc4f-4466-8eac-4f6988938dee None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:01.060630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:01.071024 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5923f-044c-46c2-af1d-bc7f7ac3bac2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:01.136184 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4968e4c-db26-4e04-934a-cff1dc29ae7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:01.137198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4968e4c-db26-4e04-934a-cff1dc29ae7b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:01.137416 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4968e4c-db26-4e04-934a-cff1dc29ae7b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:01.137713 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4968e4c-db26-4e04-934a-cff1dc29ae7b None admin] PATH_INFO: `/v3/users/9fff0f0c6da0406d81936f2fc4ca9cd9` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:01.138927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4968e4c-db26-4e04-934a-cff1dc29ae7b None admin] RBAC: Authorizing `identity:get_user(user_id=9fff0f0c6da0406d81936f2fc4ca9cd9)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:01.140381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4968e4c-db26-4e04-934a-cff1dc29ae7b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:01.142068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 188/364] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:30:00 2022] GET /identity/v3/users/9fff0f0c6da0406d81936f2fc4ca9cd9 => generated 293 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:01.142851 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4968e4c-db26-4e04-934a-cff1dc29ae7b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:01.198659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93cfb2-fc4f-4466-8eac-4f6988938dee None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:01.255753 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc79f55-9a56-4e29-a8b1-751365fbe6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:01.257202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc79f55-9a56-4e29-a8b1-751365fbe6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:01.257945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc79f55-9a56-4e29-a8b1-751365fbe6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:01.258771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc79f55-9a56-4e29-a8b1-751365fbe6e8 None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:01.260943 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc79f55-9a56-4e29-a8b1-751365fbe6e8 None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:01.263927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc79f55-9a56-4e29-a8b1-751365fbe6e8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:01.267215 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 177/365] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:30:01 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:01.269500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc79f55-9a56-4e29-a8b1-751365fbe6e8 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:01.275986 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:01.292138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4968e4c-db26-4e04-934a-cff1dc29ae7b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:01.344561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d07612-b0a4-43af-9d1a-ac283cadfcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:01.345201 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d07612-b0a4-43af-9d1a-ac283cadfcbf None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:01.345508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d07612-b0a4-43af-9d1a-ac283cadfcbf None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:01.345796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d07612-b0a4-43af-9d1a-ac283cadfcbf None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:01.346541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d07612-b0a4-43af-9d1a-ac283cadfcbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:01.346852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d07612-b0a4-43af-9d1a-ac283cadfcbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:01.348243 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d07612-b0a4-43af-9d1a-ac283cadfcbf None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:01.358116 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 189/366] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:01.358965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d07612-b0a4-43af-9d1a-ac283cadfcbf None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:01.408873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:01.435902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc79f55-9a56-4e29-a8b1-751365fbe6e8 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:01.482469 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bc55d-b554-49d3-b8ba-71715e0343e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:01.483130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bc55d-b554-49d3-b8ba-71715e0343e5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:01.483417 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bc55d-b554-49d3-b8ba-71715e0343e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:01.483693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bc55d-b554-49d3-b8ba-71715e0343e5 None admin] PATH_INFO: `/v3/groups/ebeaaf50bf9441d7992937759eff8370` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:01.485024 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bc55d-b554-49d3-b8ba-71715e0343e5 None admin] RBAC: Authorizing `identity:get_group(group_id=ebeaaf50bf9441d7992937759eff8370)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:01.486714 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bc55d-b554-49d3-b8ba-71715e0343e5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:01.488961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 178/367] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 3 10:30:01 2022] GET /identity/v3/groups/ebeaaf50bf9441d7992937759eff8370 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:01.494891 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bc55d-b554-49d3-b8ba-71715e0343e5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:01.507042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d07612-b0a4-43af-9d1a-ac283cadfcbf None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:01.555269 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d32c4e-4133-49ac-81d8-688cd1b22ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:01.556361 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d32c4e-4133-49ac-81d8-688cd1b22ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:01.556872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d32c4e-4133-49ac-81d8-688cd1b22ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:01.557371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d32c4e-4133-49ac-81d8-688cd1b22ac6 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:01.558821 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d32c4e-4133-49ac-81d8-688cd1b22ac6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:01.559388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d32c4e-4133-49ac-81d8-688cd1b22ac6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:01.562087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d32c4e-4133-49ac-81d8-688cd1b22ac6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:01.578585 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 190/368] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:01.580089 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d32c4e-4133-49ac-81d8-688cd1b22ac6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:01.594956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:01.643790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bc55d-b554-49d3-b8ba-71715e0343e5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:01.701859 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3bba5-36fd-4b08-b5f3-340536cea8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:01.702687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3bba5-36fd-4b08-b5f3-340536cea8fa None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:01.703096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3bba5-36fd-4b08-b5f3-340536cea8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:01.703501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3bba5-36fd-4b08-b5f3-340536cea8fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:01.704701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3bba5-36fd-4b08-b5f3-340536cea8fa None admin] RBAC: Adding query filter params (role.id=66ef9c3aa98247eab1c1c726f2f6a5a5, user.id=9fff0f0c6da0406d81936f2fc4ca9cd9, scope.system=all) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:01.705111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3bba5-36fd-4b08-b5f3-340536cea8fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:01.706984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3bba5-36fd-4b08-b5f3-340536cea8fa None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:01.709019 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:01.723570 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d32c4e-4133-49ac-81d8-688cd1b22ac6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:01.727121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 179/369] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Aug 3 10:30:01 2022] GET /identity/v3/role_assignments?user.id=9fff0f0c6da0406d81936f2fc4ca9cd9&scope.system=all&role.id=66ef9c3aa98247eab1c1c726f2f6a5a5 => generated 510 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:01.728369 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3bba5-36fd-4b08-b5f3-340536cea8fa None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:01.788304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433ed6b-29c8-4ea0-a71d-ef30e4caf879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:01.789810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433ed6b-29c8-4ea0-a71d-ef30e4caf879 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:01.790509 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433ed6b-29c8-4ea0-a71d-ef30e4caf879 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:01.791166 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433ed6b-29c8-4ea0-a71d-ef30e4caf879 None admin] PATH_INFO: `/v3/groups/ebeaaf50bf9441d7992937759eff8370` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:01.793718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433ed6b-29c8-4ea0-a71d-ef30e4caf879 None admin] RBAC: Authorizing `identity:get_group(group_id=ebeaaf50bf9441d7992937759eff8370)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:01.796684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433ed6b-29c8-4ea0-a71d-ef30e4caf879 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:01.800772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 191/370] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 3 10:30:01 2022] GET /identity/v3/groups/ebeaaf50bf9441d7992937759eff8370 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:01.802517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433ed6b-29c8-4ea0-a71d-ef30e4caf879 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:01.934744 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3bba5-36fd-4b08-b5f3-340536cea8fa None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:01.977028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433ed6b-29c8-4ea0-a71d-ef30e4caf879 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:01.992398 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:02.010850 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36df10-d6e9-44e1-a505-6fb6dcffe815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:02.011693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36df10-d6e9-44e1-a505-6fb6dcffe815 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:02.011994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36df10-d6e9-44e1-a505-6fb6dcffe815 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:02.013535 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36df10-d6e9-44e1-a505-6fb6dcffe815 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:02.029097 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bccbe3-e8e7-43b8-83c3-4fa170c4b597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:02.029785 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bccbe3-e8e7-43b8-83c3-4fa170c4b597 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:02.030072 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bccbe3-e8e7-43b8-83c3-4fa170c4b597 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:02.030352 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bccbe3-e8e7-43b8-83c3-4fa170c4b597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:02.031175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bccbe3-e8e7-43b8-83c3-4fa170c4b597 None admin] RBAC: Adding query filter params (group.id=ebeaaf50bf9441d7992937759eff8370, role.id=82be5cfedcb3407f8d14522ef87247ac, scope.project.id=998a52ff5527415da541925c79fddaab) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:02.031490 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bccbe3-e8e7-43b8-83c3-4fa170c4b597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:02.032957 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bccbe3-e8e7-43b8-83c3-4fa170c4b597 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:02.039801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 192/371] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 3 10:30:01 2022] GET /identity/v3/role_assignments?group.id=ebeaaf50bf9441d7992937759eff8370&scope.project.id=998a52ff5527415da541925c79fddaab&role.id=82be5cfedcb3407f8d14522ef87247ac => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:02.040657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bccbe3-e8e7-43b8-83c3-4fa170c4b597 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:02.070408 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:02.132641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36df10-d6e9-44e1-a505-6fb6dcffe815 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:02.134994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36df10-d6e9-44e1-a505-6fb6dcffe815 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:02.198620 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bccbe3-e8e7-43b8-83c3-4fa170c4b597 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:02.270351 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080b020-3aab-491a-aa2d-4059881f456a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:02.271207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080b020-3aab-491a-aa2d-4059881f456a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:02.271623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080b020-3aab-491a-aa2d-4059881f456a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:02.272060 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080b020-3aab-491a-aa2d-4059881f456a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:02.315325 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 180/372] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:02.316971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36df10-d6e9-44e1-a505-6fb6dcffe815 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:02.381884 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:02.415153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080b020-3aab-491a-aa2d-4059881f456a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:02.417232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080b020-3aab-491a-aa2d-4059881f456a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:02.467418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:02.525117 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36df10-d6e9-44e1-a505-6fb6dcffe815 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:02.574259 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607afdfb-8e60-4968-8f79-15400ef8804b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:02.574910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607afdfb-8e60-4968-8f79-15400ef8804b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:02.575197 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607afdfb-8e60-4968-8f79-15400ef8804b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:02.575480 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607afdfb-8e60-4968-8f79-15400ef8804b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:02.576301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607afdfb-8e60-4968-8f79-15400ef8804b None admin] RBAC: Adding query filter params (group.id=ebeaaf50bf9441d7992937759eff8370, role.id=82be5cfedcb3407f8d14522ef87247ac, scope.project.id=d4db61efd90f48a6b63ae8fd15240329) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:02.576656 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607afdfb-8e60-4968-8f79-15400ef8804b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:02.578359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607afdfb-8e60-4968-8f79-15400ef8804b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:02.586112 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 181/373] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 3 10:30:02 2022] GET /identity/v3/role_assignments?group.id=ebeaaf50bf9441d7992937759eff8370&scope.project.id=d4db61efd90f48a6b63ae8fd15240329&role.id=82be5cfedcb3407f8d14522ef87247ac => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:02.586994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607afdfb-8e60-4968-8f79-15400ef8804b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:02.628642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 193/374] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:02.634369 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080b020-3aab-491a-aa2d-4059881f456a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:02.722650 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:02.731159 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607afdfb-8e60-4968-8f79-15400ef8804b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:02.779322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95f30d-05df-4224-9773-ef6938964f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:02.780640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95f30d-05df-4224-9773-ef6938964f9a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:02.780937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95f30d-05df-4224-9773-ef6938964f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:02.781216 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95f30d-05df-4224-9773-ef6938964f9a None admin] PATH_INFO: `/v3/projects/a90579cdd4284a47979efcd0ca29e39a` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:02.782790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95f30d-05df-4224-9773-ef6938964f9a None admin] RBAC: Authorizing `identity:get_project(project_id=a90579cdd4284a47979efcd0ca29e39a)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:02.784719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95f30d-05df-4224-9773-ef6938964f9a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:02.787702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 182/375] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:30:02 2022] GET /identity/v3/projects/a90579cdd4284a47979efcd0ca29e39a => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:02.789168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399479b3-a1b6-4f91-b8ea-416fc59c4614 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:02.789625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399479b3-a1b6-4f91-b8ea-416fc59c4614 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:02.789912 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399479b3-a1b6-4f91-b8ea-416fc59c4614 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:02.791300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 183/376] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:02.795022 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399479b3-a1b6-4f91-b8ea-416fc59c4614 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:02.809717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080b020-3aab-491a-aa2d-4059881f456a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:02.857907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:02.884177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e1229-a62f-489c-b6da-e4e40a4bd207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:02.885228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e1229-a62f-489c-b6da-e4e40a4bd207 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:02.886031 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e1229-a62f-489c-b6da-e4e40a4bd207 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:02.886514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e1229-a62f-489c-b6da-e4e40a4bd207 None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:02.887994 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e1229-a62f-489c-b6da-e4e40a4bd207 None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:02.889715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e1229-a62f-489c-b6da-e4e40a4bd207 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:02.892836 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 194/377] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:30:02 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:30:02.894959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1e079-549e-4bbb-8739-42f96e64916d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:02.895454 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1e079-549e-4bbb-8739-42f96e64916d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:02.896236 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1e079-549e-4bbb-8739-42f96e64916d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:02.918214 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5fc1e079-549e-4bbb-8739-42f96e64916d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:02.919786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5fc1e079-549e-4bbb-8739-42f96e64916d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:02.953881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399479b3-a1b6-4f91-b8ea-416fc59c4614 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:02.996254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 195/378] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:02.997501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1e079-549e-4bbb-8739-42f96e64916d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:03.027147 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff71de3-47cf-47a8-acb6-b181ef9b55a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:03.031800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff71de3-47cf-47a8-acb6-b181ef9b55a4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:03.032382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff71de3-47cf-47a8-acb6-b181ef9b55a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:03.032858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff71de3-47cf-47a8-acb6-b181ef9b55a4 None admin] PATH_INFO: `/v3/groups/ec0cfcbbb96344a89ff8119338799d86` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:03.046117 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff71de3-47cf-47a8-acb6-b181ef9b55a4 None admin] RBAC: Authorizing `identity:get_group(group_id=ec0cfcbbb96344a89ff8119338799d86)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:03.048321 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff71de3-47cf-47a8-acb6-b181ef9b55a4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:03.052343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 184/379] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 3 10:30:02 2022] GET /identity/v3/groups/ec0cfcbbb96344a89ff8119338799d86 => generated 229 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:03.053555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff71de3-47cf-47a8-acb6-b181ef9b55a4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:03.091244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:03.208430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1e079-549e-4bbb-8739-42f96e64916d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:03.218525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff71de3-47cf-47a8-acb6-b181ef9b55a4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:03.238393 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:03.260243 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50e165-6cde-4cb2-9eab-2e955a3f4e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:03.260958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50e165-6cde-4cb2-9eab-2e955a3f4e0a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:03.261231 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50e165-6cde-4cb2-9eab-2e955a3f4e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:03.261587 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50e165-6cde-4cb2-9eab-2e955a3f4e0a None admin] PATH_INFO: `/v3/groups/ebeaaf50bf9441d7992937759eff8370` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:03.262886 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50e165-6cde-4cb2-9eab-2e955a3f4e0a None admin] RBAC: Authorizing `identity:get_group(group_id=ebeaaf50bf9441d7992937759eff8370)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:03.265324 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50e165-6cde-4cb2-9eab-2e955a3f4e0a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:03.267495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 196/380] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 3 10:30:02 2022] GET /identity/v3/groups/ebeaaf50bf9441d7992937759eff8370 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:03.268571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50e165-6cde-4cb2-9eab-2e955a3f4e0a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:03.291985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e4725-9661-4ab1-9a31-083e0ee0f947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:03.292602 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e4725-9661-4ab1-9a31-083e0ee0f947 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:03.292993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e4725-9661-4ab1-9a31-083e0ee0f947 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:03.293266 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e4725-9661-4ab1-9a31-083e0ee0f947 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:03.308403 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e4725-9661-4ab1-9a31-083e0ee0f947 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-d38e4725-9661-4ab1-9a31-083e0ee0f947 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:03.315177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:03.317457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:03.319033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:03.319033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:03.319033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:03.319033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:03.319033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:03.319033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:03.319033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:03.319033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:03.319033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:03.319033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:03.319033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:30:03.319033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:03.322187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 185/381] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:30:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:03.323208 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e4725-9661-4ab1-9a31-083e0ee0f947 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:03.442401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:03.465798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50e165-6cde-4cb2-9eab-2e955a3f4e0a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:03.478425 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e4725-9661-4ab1-9a31-083e0ee0f947 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:03.532698 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b23de-41f2-408e-985a-de670e65c890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:03.534778 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b23de-41f2-408e-985a-de670e65c890 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:03.535530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b23de-41f2-408e-985a-de670e65c890 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:03.536357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b23de-41f2-408e-985a-de670e65c890 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:03.538302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b23de-41f2-408e-985a-de670e65c890 None admin] RBAC: Adding query filter params (group.id=ec0cfcbbb96344a89ff8119338799d86, role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=a90579cdd4284a47979efcd0ca29e39a) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:03.539181 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b23de-41f2-408e-985a-de670e65c890 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:03.542383 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b23de-41f2-408e-985a-de670e65c890 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:03.553842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f39b3-c162-4d9d-beeb-4a1ba25faec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:03.554867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f39b3-c162-4d9d-beeb-4a1ba25faec2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:03.555357 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f39b3-c162-4d9d-beeb-4a1ba25faec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:03.555825 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f39b3-c162-4d9d-beeb-4a1ba25faec2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:03.557093 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 197/382] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 3 10:30:03 2022] GET /identity/v3/role_assignments?group.id=ec0cfcbbb96344a89ff8119338799d86&scope.project.id=a90579cdd4284a47979efcd0ca29e39a&role.id=01714797cab143e1a7a150c068fa9315 => generated 260 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:03.557254 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f39b3-c162-4d9d-beeb-4a1ba25faec2 None admin] RBAC: Adding query filter params (group.id=ebeaaf50bf9441d7992937759eff8370, role.id=cde4978cbb9648639ff720b5ca82886b, scope.project.id=d4db61efd90f48a6b63ae8fd15240329) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:03.557858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f39b3-c162-4d9d-beeb-4a1ba25faec2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:03.559652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b23de-41f2-408e-985a-de670e65c890 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:03.560930 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f39b3-c162-4d9d-beeb-4a1ba25faec2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:03.574390 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 186/383] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 3 10:30:03 2022] GET /identity/v3/role_assignments?group.id=ebeaaf50bf9441d7992937759eff8370&scope.project.id=d4db61efd90f48a6b63ae8fd15240329&role.id=cde4978cbb9648639ff720b5ca82886b => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:03.707544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:03.807220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b23de-41f2-408e-985a-de670e65c890 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:03.883272 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec4c05-6359-4a7c-bb62-519fc8eb12cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:03.884258 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec4c05-6359-4a7c-bb62-519fc8eb12cf None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:03.884759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec4c05-6359-4a7c-bb62-519fc8eb12cf None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:03.885236 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec4c05-6359-4a7c-bb62-519fc8eb12cf None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:03.886727 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec4c05-6359-4a7c-bb62-519fc8eb12cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:03.887296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec4c05-6359-4a7c-bb62-519fc8eb12cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:03.889915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec4c05-6359-4a7c-bb62-519fc8eb12cf None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:03.906658 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 198/384] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:03.913121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f39b3-c162-4d9d-beeb-4a1ba25faec2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:04.108823 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f39b3-c162-4d9d-beeb-4a1ba25faec2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:04.174506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4d143-b49e-4a90-adfb-9d5c91dfee75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:04.175732 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4d143-b49e-4a90-adfb-9d5c91dfee75 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:04.176227 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4d143-b49e-4a90-adfb-9d5c91dfee75 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:04.176708 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4d143-b49e-4a90-adfb-9d5c91dfee75 None admin] PATH_INFO: `/v3/groups/ebeaaf50bf9441d7992937759eff8370` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:04.179201 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4d143-b49e-4a90-adfb-9d5c91dfee75 None admin] RBAC: Authorizing `identity:get_group(group_id=ebeaaf50bf9441d7992937759eff8370)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:04.181956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4d143-b49e-4a90-adfb-9d5c91dfee75 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:04.185126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 187/385] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 3 10:30:03 2022] GET /identity/v3/groups/ebeaaf50bf9441d7992937759eff8370 => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:04.192997 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec4c05-6359-4a7c-bb62-519fc8eb12cf None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:04.405097 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec4c05-6359-4a7c-bb62-519fc8eb12cf None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:04.436208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:04.475797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072ff9f-f75d-4313-bedd-c7053f1ed2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:04.476766 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072ff9f-f75d-4313-bedd-c7053f1ed2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:04.476766 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072ff9f-f75d-4313-bedd-c7053f1ed2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:04.476993 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072ff9f-f75d-4313-bedd-c7053f1ed2a1 None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:04.478239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072ff9f-f75d-4313-bedd-c7053f1ed2a1 None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:04.479718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072ff9f-f75d-4313-bedd-c7053f1ed2a1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:04.481499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 199/386] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:30:04 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:04.487804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e489e87-4b62-43c9-8e38-9237910f19f9 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:04.488042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072ff9f-f75d-4313-bedd-c7053f1ed2a1 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:04.488228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e489e87-4b62-43c9-8e38-9237910f19f9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:04.488538 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e489e87-4b62-43c9-8e38-9237910f19f9 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:04.490187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 188/387] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:04.497119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e547590-f920-4baa-8a53-9add48d14e7a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:04.497530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e547590-f920-4baa-8a53-9add48d14e7a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:04.497715 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e547590-f920-4baa-8a53-9add48d14e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:04.519778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-1e547590-f920-4baa-8a53-9add48d14e7a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:04.521360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1e547590-f920-4baa-8a53-9add48d14e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:04.535032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:04.606592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 189/388] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:30:04.632403 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e547590-f920-4baa-8a53-9add48d14e7a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:04.734428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072ff9f-f75d-4313-bedd-c7053f1ed2a1 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:04.803669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2231dd1-f257-475b-ba48-d3eeccb7e5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:04.804377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2231dd1-f257-475b-ba48-d3eeccb7e5e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:04.804655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2231dd1-f257-475b-ba48-d3eeccb7e5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:04.804938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2231dd1-f257-475b-ba48-d3eeccb7e5e5 None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab/groups/ebeaaf50bf9441d7992937759eff8370/roles/cde4978cbb9648639ff720b5ca82886b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:04.811663 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2231dd1-f257-475b-ba48-d3eeccb7e5e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=998a52ff5527415da541925c79fddaab, group_id=ebeaaf50bf9441d7992937759eff8370, role_id=cde4978cbb9648639ff720b5ca82886b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:04.813191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2231dd1-f257-475b-ba48-d3eeccb7e5e5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:04.823232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e547590-f920-4baa-8a53-9add48d14e7a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:04.845423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 200/389] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Aug 3 10:30:04 2022] PUT /identity/v3/projects/998a52ff5527415da541925c79fddaab/groups/ebeaaf50bf9441d7992937759eff8370/roles/cde4978cbb9648639ff720b5ca82886b => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:30:04.903144 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cd159-87f5-4f15-b64b-dd86ebba1b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:04.904315 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cd159-87f5-4f15-b64b-dd86ebba1b5f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:04.904788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cd159-87f5-4f15-b64b-dd86ebba1b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:04.905242 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cd159-87f5-4f15-b64b-dd86ebba1b5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:04.936785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49cd159-87f5-4f15-b64b-dd86ebba1b5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-e49cd159-87f5-4f15-b64b-dd86ebba1b5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:04.950332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:04.951332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:04.952123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:04.952123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:04.952123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:04.952123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:04.952123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:04.952123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:04.952123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:04.952123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:04.952123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:04.952123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:04.952123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:30:04.952123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:04.953278 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 190/390] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:04.960909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2231dd1-f257-475b-ba48-d3eeccb7e5e5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:05.044133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:05.162683 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2231dd1-f257-475b-ba48-d3eeccb7e5e5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:05.251246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ffcdd-1cfe-44f3-b329-92aaf03f15ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:05.252347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ffcdd-1cfe-44f3-b329-92aaf03f15ad None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:05.252850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ffcdd-1cfe-44f3-b329-92aaf03f15ad None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:05.253386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ffcdd-1cfe-44f3-b329-92aaf03f15ad None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:05.254795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ffcdd-1cfe-44f3-b329-92aaf03f15ad None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:05.255359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ffcdd-1cfe-44f3-b329-92aaf03f15ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:05.258023 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ffcdd-1cfe-44f3-b329-92aaf03f15ad None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:05.286393 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 201/391] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:30:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:05.292405 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cd159-87f5-4f15-b64b-dd86ebba1b5f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:05.306038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:05.436613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cd159-87f5-4f15-b64b-dd86ebba1b5f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:05.475788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f416a-a433-4d5d-98cb-a92c78f12adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:05.476468 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f416a-a433-4d5d-98cb-a92c78f12adc None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:05.476750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f416a-a433-4d5d-98cb-a92c78f12adc None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:05.477024 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f416a-a433-4d5d-98cb-a92c78f12adc None admin] PATH_INFO: `/v3/groups/ebeaaf50bf9441d7992937759eff8370` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:05.478262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f416a-a433-4d5d-98cb-a92c78f12adc None admin] RBAC: Authorizing `identity:get_group(group_id=ebeaaf50bf9441d7992937759eff8370)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:05.479738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f416a-a433-4d5d-98cb-a92c78f12adc None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:05.481771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 191/392] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 3 10:30:05 2022] GET /identity/v3/groups/ebeaaf50bf9441d7992937759eff8370 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:05.488343 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ffcdd-1cfe-44f3-b329-92aaf03f15ad None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:05.715960 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ffcdd-1cfe-44f3-b329-92aaf03f15ad None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:05.720627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:05.796458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecabb13-0467-4fef-bc53-f1c587b79676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:05.797626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecabb13-0467-4fef-bc53-f1c587b79676 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:05.798108 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecabb13-0467-4fef-bc53-f1c587b79676 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:05.798604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecabb13-0467-4fef-bc53-f1c587b79676 None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:05.801619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecabb13-0467-4fef-bc53-f1c587b79676 None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:05.804564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecabb13-0467-4fef-bc53-f1c587b79676 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:05.808239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 202/393] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:30:05 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:05.813906 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f416a-a433-4d5d-98cb-a92c78f12adc None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:05.919390 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:05.948241 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb65eeb-555d-479e-9490-c3d058e823a9 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:05.948673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb65eeb-555d-479e-9490-c3d058e823a9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:05.949164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb65eeb-555d-479e-9490-c3d058e823a9 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:05.952306 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 203/394] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:05.952552 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f416a-a433-4d5d-98cb-a92c78f12adc None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:05.960373 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800fe73-d334-44e6-b5e5-49bf9d543dc9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:05.960875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800fe73-d334-44e6-b5e5-49bf9d543dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:05.961370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800fe73-d334-44e6-b5e5-49bf9d543dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:06.002309 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9aa33c-278d-4b64-b2ee-b9f46b699efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:06.003090 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9aa33c-278d-4b64-b2ee-b9f46b699efe None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:06.003361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9aa33c-278d-4b64-b2ee-b9f46b699efe None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:06.003643 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9aa33c-278d-4b64-b2ee-b9f46b699efe None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab/groups/ebeaaf50bf9441d7992937759eff8370/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:06.005877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9aa33c-278d-4b64-b2ee-b9f46b699efe None admin] RBAC: Authorizing `identity:create_grant(project_id=998a52ff5527415da541925c79fddaab, group_id=ebeaaf50bf9441d7992937759eff8370, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:06.007340 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9aa33c-278d-4b64-b2ee-b9f46b699efe None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:06.008200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0800fe73-d334-44e6-b5e5-49bf9d543dc9 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:06.010329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0800fe73-d334-44e6-b5e5-49bf9d543dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:06.050700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 192/395] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Aug 3 10:30:05 2022] PUT /identity/v3/projects/998a52ff5527415da541925c79fddaab/groups/ebeaaf50bf9441d7992937759eff8370/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:30:06.052114 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c0db4-7aef-4cf8-96ec-0871d1b21056 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:06.052405 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c0db4-7aef-4cf8-96ec-0871d1b21056 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:06.052680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c0db4-7aef-4cf8-96ec-0871d1b21056 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:06.054582 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 193/396] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:06.067508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadef29-2726-4dd6-ab7d-f76b2a225033 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:06.067829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadef29-2726-4dd6-ab7d-f76b2a225033 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:06.068057 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadef29-2726-4dd6-ab7d-f76b2a225033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:06.101217 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ffadef29-2726-4dd6-ab7d-f76b2a225033 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:06.102635 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ffadef29-2726-4dd6-ab7d-f76b2a225033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:06.132762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 204/397] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:06.183459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800fe73-d334-44e6-b5e5-49bf9d543dc9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:06.226678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 194/398] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:30:06.266431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadef29-2726-4dd6-ab7d-f76b2a225033 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:06.359529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800fe73-d334-44e6-b5e5-49bf9d543dc9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:06.361591 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:06.371495 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:06.409780 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56edc1-2427-4c80-bf38-38a6aef3a582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:06.411136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56edc1-2427-4c80-bf38-38a6aef3a582 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:06.412212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56edc1-2427-4c80-bf38-38a6aef3a582 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:06.412688 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56edc1-2427-4c80-bf38-38a6aef3a582 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:06.429302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56edc1-2427-4c80-bf38-38a6aef3a582 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:06.432822 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadef29-2726-4dd6-ab7d-f76b2a225033 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-2a56edc1-2427-4c80-bf38-38a6aef3a582 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:06.435044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:06.435945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:06.436736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:06.436736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:06.436736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:06.436736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:06.436736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:06.436736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:06.436736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:06.436736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:06.436736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:06.436736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:06.436736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:30:06.436736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:06.439653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 205/399] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:30:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:30:06.450545 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56edc1-2427-4c80-bf38-38a6aef3a582 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:06.480055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da721a-5641-4d74-84ee-d50b68453f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:06.480751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da721a-5641-4d74-84ee-d50b68453f69 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:06.481134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da721a-5641-4d74-84ee-d50b68453f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:06.481457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da721a-5641-4d74-84ee-d50b68453f69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:06.494147 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da721a-5641-4d74-84ee-d50b68453f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-73da721a-5641-4d74-84ee-d50b68453f69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:06.500026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:06.501251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:06.502297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:06.502297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:06.502297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:06.502297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:06.502297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:06.502297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:06.502297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:06.502297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:06.502297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:06.502297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:06.502297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:30:06.502297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:06.502297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 195/400] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:30:06.506857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da721a-5641-4d74-84ee-d50b68453f69 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:06.642602 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da721a-5641-4d74-84ee-d50b68453f69 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:06.673915 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:06.689010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5eb6d-5611-4b13-acea-17877889a471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:06.689779 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5eb6d-5611-4b13-acea-17877889a471 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:06.690136 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5eb6d-5611-4b13-acea-17877889a471 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:06.691051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5eb6d-5611-4b13-acea-17877889a471 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:06.692115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5eb6d-5611-4b13-acea-17877889a471 None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:06.692482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5eb6d-5611-4b13-acea-17877889a471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:06.694033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5eb6d-5611-4b13-acea-17877889a471 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:06.708801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 196/401] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:30:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:06.710849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17539d78-f631-4bdc-a6dd-aca0440ddbd2 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:06.711522 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17539d78-f631-4bdc-a6dd-aca0440ddbd2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:06.712045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17539d78-f631-4bdc-a6dd-aca0440ddbd2 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:06.714979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 197/402] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:06.716000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56edc1-2427-4c80-bf38-38a6aef3a582 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:06.716701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17539d78-f631-4bdc-a6dd-aca0440ddbd2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:06.758627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42291bc3-bb7e-483c-bb48-1c2a9d176bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:06.759167 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42291bc3-bb7e-483c-bb48-1c2a9d176bec None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:06.759447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42291bc3-bb7e-483c-bb48-1c2a9d176bec None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:06.759718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42291bc3-bb7e-483c-bb48-1c2a9d176bec None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:06.760451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42291bc3-bb7e-483c-bb48-1c2a9d176bec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:06.760754 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42291bc3-bb7e-483c-bb48-1c2a9d176bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:06.762468 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42291bc3-bb7e-483c-bb48-1c2a9d176bec None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:06.772105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 206/403] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:06.776824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59cb49-f809-472b-9087-c08cb232191e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:06.776824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59cb49-f809-472b-9087-c08cb232191e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:06.776824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59cb49-f809-472b-9087-c08cb232191e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:06.793604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5f59cb49-f809-472b-9087-c08cb232191e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:06.794917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5f59cb49-f809-472b-9087-c08cb232191e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:06.816061 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:06.865789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17539d78-f631-4bdc-a6dd-aca0440ddbd2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:06.882696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 207/404] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:06.883604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59cb49-f809-472b-9087-c08cb232191e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:06.925733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e33303-973e-4eb7-bc9c-669a8633b9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:06.926419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e33303-973e-4eb7-bc9c-669a8633b9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:06.926700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e33303-973e-4eb7-bc9c-669a8633b9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:06.926974 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e33303-973e-4eb7-bc9c-669a8633b9b0 None admin] PATH_INFO: `/v3/groups/ebeaaf50bf9441d7992937759eff8370` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:06.928205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e33303-973e-4eb7-bc9c-669a8633b9b0 None admin] RBAC: Authorizing `identity:get_group(group_id=ebeaaf50bf9441d7992937759eff8370)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:06.929705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e33303-973e-4eb7-bc9c-669a8633b9b0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:06.936748 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 198/405] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 3 10:30:06 2022] GET /identity/v3/groups/ebeaaf50bf9441d7992937759eff8370 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:06.937965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e33303-973e-4eb7-bc9c-669a8633b9b0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:06.999329 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:07.007213 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59cb49-f809-472b-9087-c08cb232191e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:07.048371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e33303-973e-4eb7-bc9c-669a8633b9b0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:07.049317 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad131c8d-b22f-4e4b-8b32-4086fc8ddac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:07.050670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad131c8d-b22f-4e4b-8b32-4086fc8ddac4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:07.050955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad131c8d-b22f-4e4b-8b32-4086fc8ddac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:07.051318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad131c8d-b22f-4e4b-8b32-4086fc8ddac4 None admin] PATH_INFO: `/v3/groups/ec0cfcbbb96344a89ff8119338799d86` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:07.052552 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad131c8d-b22f-4e4b-8b32-4086fc8ddac4 None admin] RBAC: Authorizing `identity:get_group(group_id=ec0cfcbbb96344a89ff8119338799d86)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:07.054094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad131c8d-b22f-4e4b-8b32-4086fc8ddac4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:07.056468 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 208/406] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 3 10:30:06 2022] GET /identity/v3/groups/ec0cfcbbb96344a89ff8119338799d86 => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:07.057399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad131c8d-b22f-4e4b-8b32-4086fc8ddac4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:07.085956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:07.086297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b26d28-fe02-43d9-b744-17f341bb32bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:07.087065 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b26d28-fe02-43d9-b744-17f341bb32bf None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:07.087245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b26d28-fe02-43d9-b744-17f341bb32bf None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:07.087586 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b26d28-fe02-43d9-b744-17f341bb32bf None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:07.089227 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b26d28-fe02-43d9-b744-17f341bb32bf None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:07.090632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b26d28-fe02-43d9-b744-17f341bb32bf None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:07.092355 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 199/407] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:30:06 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:30:07.093373 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b26d28-fe02-43d9-b744-17f341bb32bf None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:07.180219 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad131c8d-b22f-4e4b-8b32-4086fc8ddac4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:07.207959 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b26d28-fe02-43d9-b744-17f341bb32bf None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:07.228846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186990c-0de8-4eb8-b5b5-e89b0c43324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:07.229864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186990c-0de8-4eb8-b5b5-e89b0c43324c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:07.230286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186990c-0de8-4eb8-b5b5-e89b0c43324c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:07.230700 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186990c-0de8-4eb8-b5b5-e89b0c43324c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:07.234559 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:07.245876 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4186990c-0de8-4eb8-b5b5-e89b0c43324c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:07.246653 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8cf97-b1d9-4c5e-bd6d-88295df7e881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:07.247389 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8cf97-b1d9-4c5e-bd6d-88295df7e881 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:07.247670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8cf97-b1d9-4c5e-bd6d-88295df7e881 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:07.247947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8cf97-b1d9-4c5e-bd6d-88295df7e881 None admin] PATH_INFO: `/v3/projects/a90579cdd4284a47979efcd0ca29e39a` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:07.249151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8cf97-b1d9-4c5e-bd6d-88295df7e881 None admin] RBAC: Authorizing `identity:get_project(project_id=a90579cdd4284a47979efcd0ca29e39a)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:07.250693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8cf97-b1d9-4c5e-bd6d-88295df7e881 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-4186990c-0de8-4eb8-b5b5-e89b0c43324c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:07.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:07.253917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:07.254900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:07.254900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:07.254900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:07.254900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:07.254900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:07.254900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:07.254900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:07.254900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:07.254900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:07.254900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:07.254900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:30:07.254900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:07.255498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 200/408] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:30:07 2022] GET /identity/v3/projects/a90579cdd4284a47979efcd0ca29e39a => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:07.255498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8cf97-b1d9-4c5e-bd6d-88295df7e881 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:07.257700 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 209/409] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:30:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:07.259419 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186990c-0de8-4eb8-b5b5-e89b0c43324c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:07.373238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8cf97-b1d9-4c5e-bd6d-88295df7e881 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:07.410362 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:07.431911 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8f81f-993d-4f63-a5e7-115541dd6a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:07.432999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8f81f-993d-4f63-a5e7-115541dd6a71 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:07.433646 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8f81f-993d-4f63-a5e7-115541dd6a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:07.434137 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8f81f-993d-4f63-a5e7-115541dd6a71 None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329/groups/ebeaaf50bf9441d7992937759eff8370/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:07.434745 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186990c-0de8-4eb8-b5b5-e89b0c43324c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:07.438391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8f81f-993d-4f63-a5e7-115541dd6a71 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4db61efd90f48a6b63ae8fd15240329, group_id=ebeaaf50bf9441d7992937759eff8370, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:07.441064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8f81f-993d-4f63-a5e7-115541dd6a71 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:07.468367 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 201/410] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Aug 3 10:30:07 2022] PUT /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329/groups/ebeaaf50bf9441d7992937759eff8370/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:30:07.475499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8f81f-993d-4f63-a5e7-115541dd6a71 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:07.481804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519c823-6704-4149-b3d4-076beccb4503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:07.482862 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519c823-6704-4149-b3d4-076beccb4503 None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:07.483622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519c823-6704-4149-b3d4-076beccb4503 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:07.484485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519c823-6704-4149-b3d4-076beccb4503 None admin] PATH_INFO: `/v3/projects/a90579cdd4284a47979efcd0ca29e39a/groups/ec0cfcbbb96344a89ff8119338799d86/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:07.494025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519c823-6704-4149-b3d4-076beccb4503 None admin] RBAC: Authorizing `identity:create_grant(project_id=a90579cdd4284a47979efcd0ca29e39a, group_id=ec0cfcbbb96344a89ff8119338799d86, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:07.495910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519c823-6704-4149-b3d4-076beccb4503 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:07.522417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 210/411] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Aug 3 10:30:07 2022] PUT /identity/v3/projects/a90579cdd4284a47979efcd0ca29e39a/groups/ec0cfcbbb96344a89ff8119338799d86/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:30:07.575904 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:07.640845 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8f81f-993d-4f63-a5e7-115541dd6a71 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:07.690452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1229d3-c27e-4162-9318-d6d742b45f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:07.691648 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1229d3-c27e-4162-9318-d6d742b45f3c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:07.691939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1229d3-c27e-4162-9318-d6d742b45f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:07.692221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1229d3-c27e-4162-9318-d6d742b45f3c None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:07.695330 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1229d3-c27e-4162-9318-d6d742b45f3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:07.695640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1229d3-c27e-4162-9318-d6d742b45f3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:07.697250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1229d3-c27e-4162-9318-d6d742b45f3c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:07.719727 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 202/412] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:07.729144 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519c823-6704-4149-b3d4-076beccb4503 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:07.949642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519c823-6704-4149-b3d4-076beccb4503 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:08.009784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ed539-e141-4301-a7dd-6fb33b7cc5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:08.010931 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ed539-e141-4301-a7dd-6fb33b7cc5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:08.011422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ed539-e141-4301-a7dd-6fb33b7cc5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:08.011966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ed539-e141-4301-a7dd-6fb33b7cc5b8 None admin] PATH_INFO: `/v3/groups/ebeaaf50bf9441d7992937759eff8370` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:08.014411 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ed539-e141-4301-a7dd-6fb33b7cc5b8 None admin] RBAC: Authorizing `identity:get_group(group_id=ebeaaf50bf9441d7992937759eff8370)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:08.017146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ed539-e141-4301-a7dd-6fb33b7cc5b8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:08.021253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 211/413] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 3 10:30:07 2022] GET /identity/v3/groups/ebeaaf50bf9441d7992937759eff8370 => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:08.029873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1229d3-c27e-4162-9318-d6d742b45f3c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:08.168067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1229d3-c27e-4162-9318-d6d742b45f3c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:08.189231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:08.211211 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133eb9f-c00a-403d-a591-1f4305800c4b None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:08.211616 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133eb9f-c00a-403d-a591-1f4305800c4b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:08.212113 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133eb9f-c00a-403d-a591-1f4305800c4b None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:08.220224 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 212/414] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:08.223396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a50c9c-3397-460c-9ed2-7017348a33f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:08.224089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a50c9c-3397-460c-9ed2-7017348a33f8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:08.224370 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a50c9c-3397-460c-9ed2-7017348a33f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:08.224649 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a50c9c-3397-460c-9ed2-7017348a33f8 None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:08.225383 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc900fd9-e591-4fe4-8231-17a871cf0eda None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:08.225878 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc900fd9-e591-4fe4-8231-17a871cf0eda None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:08.226354 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc900fd9-e591-4fe4-8231-17a871cf0eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:08.230377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a50c9c-3397-460c-9ed2-7017348a33f8 None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:08.232105 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a50c9c-3397-460c-9ed2-7017348a33f8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:08.234365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 203/415] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:30:08 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:30:08.247233 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a50c9c-3397-460c-9ed2-7017348a33f8 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:08.264599 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-bc900fd9-e591-4fe4-8231-17a871cf0eda None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:08.266923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bc900fd9-e591-4fe4-8231-17a871cf0eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:08.285701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:08.483399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 213/416] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:30:08.486078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a50c9c-3397-460c-9ed2-7017348a33f8 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:08.527382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc900fd9-e591-4fe4-8231-17a871cf0eda None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:08.554039 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8415ca-1212-4121-b1cc-3cb7dcabe99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:08.554742 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8415ca-1212-4121-b1cc-3cb7dcabe99b None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:08.555025 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8415ca-1212-4121-b1cc-3cb7dcabe99b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:08.555301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8415ca-1212-4121-b1cc-3cb7dcabe99b None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329/groups/ebeaaf50bf9441d7992937759eff8370/roles/cde4978cbb9648639ff720b5ca82886b` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:08.560143 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8415ca-1212-4121-b1cc-3cb7dcabe99b None admin] RBAC: Authorizing `identity:create_grant(project_id=d4db61efd90f48a6b63ae8fd15240329, group_id=ebeaaf50bf9441d7992937759eff8370, role_id=cde4978cbb9648639ff720b5ca82886b)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:08.561645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8415ca-1212-4121-b1cc-3cb7dcabe99b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:08.592149 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 204/417] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Aug 3 10:30:08 2022] PUT /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329/groups/ebeaaf50bf9441d7992937759eff8370/roles/cde4978cbb9648639ff720b5ca82886b => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:30:08.673726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc900fd9-e591-4fe4-8231-17a871cf0eda None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:08.725567 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff0114-9423-48fa-8867-c4a12bf23d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:08.726258 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff0114-9423-48fa-8867-c4a12bf23d71 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:08.726539 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff0114-9423-48fa-8867-c4a12bf23d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:08.726816 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff0114-9423-48fa-8867-c4a12bf23d71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:08.740963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:08.744440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff0114-9423-48fa-8867-c4a12bf23d71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-2dff0114-9423-48fa-8867-c4a12bf23d71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:08.754438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:08.756280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:08.758759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:08.758759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:08.758759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:08.758759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:08.758759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:08.758759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:08.758759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:08.758759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:08.758759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:08.758759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:08.758759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:30:08.758759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:08.758759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 214/418] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:30:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:08.771658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8415ca-1212-4121-b1cc-3cb7dcabe99b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:08.792880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047de990-ea2e-4176-9a87-95d9e6813a76 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:08.793202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047de990-ea2e-4176-9a87-95d9e6813a76 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:08.793570 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047de990-ea2e-4176-9a87-95d9e6813a76 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:08.795222 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 215/419] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:08.801234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a7921-2cd8-4b5f-9fbd-60e3094e7b22 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:08.801546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a7921-2cd8-4b5f-9fbd-60e3094e7b22 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:08.801852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a7921-2cd8-4b5f-9fbd-60e3094e7b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:08.827099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-cf0a7921-2cd8-4b5f-9fbd-60e3094e7b22 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:08.828491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-cf0a7921-2cd8-4b5f-9fbd-60e3094e7b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:08.847990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:08.918986 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 216/420] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:08.925168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8415ca-1212-4121-b1cc-3cb7dcabe99b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:08.944975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a7921-2cd8-4b5f-9fbd-60e3094e7b22 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:08.988026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0ec4e-0a47-45fa-ad3c-b1a926b3a9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:08.988789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0ec4e-0a47-45fa-ad3c-b1a926b3a9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:08.989077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0ec4e-0a47-45fa-ad3c-b1a926b3a9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:08.989407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0ec4e-0a47-45fa-ad3c-b1a926b3a9e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:08.990317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0ec4e-0a47-45fa-ad3c-b1a926b3a9e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:08.990750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0ec4e-0a47-45fa-ad3c-b1a926b3a9e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:09.000997 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0ec4e-0a47-45fa-ad3c-b1a926b3a9e5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:09.008671 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 205/421] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:30:09.016726 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0ec4e-0a47-45fa-ad3c-b1a926b3a9e5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:09.176815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a7921-2cd8-4b5f-9fbd-60e3094e7b22 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:09.217878 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:09.240537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e379fdc-aa2d-4552-8a3f-20ce2da0a525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:09.240537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e379fdc-aa2d-4552-8a3f-20ce2da0a525 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:09.240537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e379fdc-aa2d-4552-8a3f-20ce2da0a525 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:09.240537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e379fdc-aa2d-4552-8a3f-20ce2da0a525 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:09.268860 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e379fdc-aa2d-4552-8a3f-20ce2da0a525 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-8e379fdc-aa2d-4552-8a3f-20ce2da0a525 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:09.274413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:09.275366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:09.276254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:09.276254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:09.276254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:09.276254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:09.276254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:09.276254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:09.276254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:09.276254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:09.276254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:09.276254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:09.276254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:30:09.276254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:09.288905 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 217/422] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:30:09.289919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e379fdc-aa2d-4552-8a3f-20ce2da0a525 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:09.312589 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0ec4e-0a47-45fa-ad3c-b1a926b3a9e5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:09.314840 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:09.396928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09e684-eef6-4dfe-a830-d375a89cc73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:09.397608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09e684-eef6-4dfe-a830-d375a89cc73e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:09.397898 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09e684-eef6-4dfe-a830-d375a89cc73e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:09.398180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09e684-eef6-4dfe-a830-d375a89cc73e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:09.496710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e379fdc-aa2d-4552-8a3f-20ce2da0a525 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:09.584486 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c877da-d24c-472f-9fa5-764c36d41cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:09.585450 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c877da-d24c-472f-9fa5-764c36d41cdd None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:09.585950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c877da-d24c-472f-9fa5-764c36d41cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:09.587647 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c877da-d24c-472f-9fa5-764c36d41cdd None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:09.588857 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c877da-d24c-472f-9fa5-764c36d41cdd None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:09.591427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c877da-d24c-472f-9fa5-764c36d41cdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:09.593531 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c877da-d24c-472f-9fa5-764c36d41cdd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:09.594574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09e684-eef6-4dfe-a830-d375a89cc73e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:09.596278 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09e684-eef6-4dfe-a830-d375a89cc73e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:09.617005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 218/423] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:30:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:09.621425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c877da-d24c-472f-9fa5-764c36d41cdd None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:09.746032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:09.854747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 206/424] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 834 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:09.864171 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c877da-d24c-472f-9fa5-764c36d41cdd None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:09.868512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09e684-eef6-4dfe-a830-d375a89cc73e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:09.870855 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:09.928859 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f879c-d36d-4af9-a10d-490053ee9d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:09.931872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f879c-d36d-4af9-a10d-490053ee9d49 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:09.931872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f879c-d36d-4af9-a10d-490053ee9d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:09.931872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f879c-d36d-4af9-a10d-490053ee9d49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:10.112775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09e684-eef6-4dfe-a830-d375a89cc73e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:10.165498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652fd2a-5a9d-43e9-a029-65d8d3029714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:10.165498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652fd2a-5a9d-43e9-a029-65d8d3029714 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:10.165498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652fd2a-5a9d-43e9-a029-65d8d3029714 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:10.165498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652fd2a-5a9d-43e9-a029-65d8d3029714 None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:10.167235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652fd2a-5a9d-43e9-a029-65d8d3029714 None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:10.167966 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652fd2a-5a9d-43e9-a029-65d8d3029714 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:10.173875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f879c-d36d-4af9-a10d-490053ee9d49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:10.174704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 207/425] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:30:09 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:30:10.175903 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f879c-d36d-4af9-a10d-490053ee9d49 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:10.176108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093cdbf-58af-48d0-a7f9-2411e7a22314 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:10.176393 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093cdbf-58af-48d0-a7f9-2411e7a22314 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:10.176663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093cdbf-58af-48d0-a7f9-2411e7a22314 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:10.182673 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 208/426] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:10.182673 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093cdbf-58af-48d0-a7f9-2411e7a22314 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:10.257741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:10.265840 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:10.303567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093cdbf-58af-48d0-a7f9-2411e7a22314 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:10.344108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0334368-717e-4dd3-adab-73833a694cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:10.344741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 219/427] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:10.345713 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0334368-717e-4dd3-adab-73833a694cdf None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:10.345894 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a6be0-31e2-4d7b-94db-8566e3c6d2ff None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:10.346130 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a6be0-31e2-4d7b-94db-8566e3c6d2ff None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:10.346406 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a6be0-31e2-4d7b-94db-8566e3c6d2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:10.346837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0334368-717e-4dd3-adab-73833a694cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:10.347310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0334368-717e-4dd3-adab-73833a694cdf None admin] PATH_INFO: `/v3/groups/ebeaaf50bf9441d7992937759eff8370` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:10.348829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0334368-717e-4dd3-adab-73833a694cdf None admin] RBAC: Authorizing `identity:get_group(group_id=ebeaaf50bf9441d7992937759eff8370)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:10.350720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0334368-717e-4dd3-adab-73833a694cdf None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:10.353015 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 209/428] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 3 10:30:10 2022] GET /identity/v3/groups/ebeaaf50bf9441d7992937759eff8370 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:10.354434 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0334368-717e-4dd3-adab-73833a694cdf None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:10.374487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2f9a6be0-31e2-4d7b-94db-8566e3c6d2ff None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:10.376048 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2f9a6be0-31e2-4d7b-94db-8566e3c6d2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:10.473264 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 220/429] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:10.475053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a6be0-31e2-4d7b-94db-8566e3c6d2ff None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:10.513601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0334368-717e-4dd3-adab-73833a694cdf None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:10.555262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d8f36-2c85-4f3f-ba26-2bfb765b695a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:10.556056 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d8f36-2c85-4f3f-ba26-2bfb765b695a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:10.556408 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d8f36-2c85-4f3f-ba26-2bfb765b695a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:10.556756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d8f36-2c85-4f3f-ba26-2bfb765b695a None admin] PATH_INFO: `/v3/projects/998a52ff5527415da541925c79fddaab` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:10.558192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d8f36-2c85-4f3f-ba26-2bfb765b695a None admin] RBAC: Authorizing `identity:get_project(project_id=998a52ff5527415da541925c79fddaab)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:10.559732 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d8f36-2c85-4f3f-ba26-2bfb765b695a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:10.561786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 210/430] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:30:10 2022] GET /identity/v3/projects/998a52ff5527415da541925c79fddaab => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:10.562470 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d8f36-2c85-4f3f-ba26-2bfb765b695a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:10.572798 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:10.629419 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:10.659318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a6be0-31e2-4d7b-94db-8566e3c6d2ff None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:10.715595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec38900-c489-48c4-8d2e-4092524fcbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:10.716580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec38900-c489-48c4-8d2e-4092524fcbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:10.717045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec38900-c489-48c4-8d2e-4092524fcbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:10.717539 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec38900-c489-48c4-8d2e-4092524fcbc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:10.718752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec38900-c489-48c4-8d2e-4092524fcbc1 None admin] RBAC: Adding query filter params (group.id=ebeaaf50bf9441d7992937759eff8370, role.id=cde4978cbb9648639ff720b5ca82886b, scope.project.id=998a52ff5527415da541925c79fddaab) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:10.719265 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec38900-c489-48c4-8d2e-4092524fcbc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:10.721847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec38900-c489-48c4-8d2e-4092524fcbc1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:10.722998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d8f36-2c85-4f3f-ba26-2bfb765b695a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:10.733732 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 221/431] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 3 10:30:10 2022] GET /identity/v3/role_assignments?group.id=ebeaaf50bf9441d7992937759eff8370&scope.project.id=998a52ff5527415da541925c79fddaab&role.id=cde4978cbb9648639ff720b5ca82886b => generated 611 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:10.735765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7693819-93c3-41bd-b075-3078a00dbead None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:10.736236 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7693819-93c3-41bd-b075-3078a00dbead None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:10.736703 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7693819-93c3-41bd-b075-3078a00dbead None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:10.739824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 222/432] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:10.740522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7693819-93c3-41bd-b075-3078a00dbead None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:10.801659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec3f99-bee9-484d-986d-105400f154f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:10.803240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec3f99-bee9-484d-986d-105400f154f2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:10.803947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec3f99-bee9-484d-986d-105400f154f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:10.804653 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec3f99-bee9-484d-986d-105400f154f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:10.825802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec3f99-bee9-484d-986d-105400f154f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-6cec3f99-bee9-484d-986d-105400f154f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:10.834955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:10.836558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:10.839058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:10.839058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:10.839058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:10.839058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:10.839058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:10.839058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:10.839058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:10.839058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:10.839058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:10.839058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:10.839058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 10:30:10.839058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:10.842671 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 211/433] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:10.845320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9da6a-3855-485b-9383-0ec20d144661 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:10.846087 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9da6a-3855-485b-9383-0ec20d144661 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:10.847316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9da6a-3855-485b-9383-0ec20d144661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:10.886024 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2bb9da6a-3855-485b-9383-0ec20d144661 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:10.888361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2bb9da6a-3855-485b-9383-0ec20d144661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:10.961366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7693819-93c3-41bd-b075-3078a00dbead None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:11.011958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37736be7-226c-4a3d-b2d9-6323b12a7db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:11.013059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37736be7-226c-4a3d-b2d9-6323b12a7db8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:11.013588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37736be7-226c-4a3d-b2d9-6323b12a7db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:11.014047 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37736be7-226c-4a3d-b2d9-6323b12a7db8 None admin] PATH_INFO: `/v3/groups/ebeaaf50bf9441d7992937759eff8370` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:11.014870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:11.016943 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37736be7-226c-4a3d-b2d9-6323b12a7db8 None admin] RBAC: Authorizing `identity:get_group(group_id=ebeaaf50bf9441d7992937759eff8370)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:11.018339 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37736be7-226c-4a3d-b2d9-6323b12a7db8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:11.020748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 223/434] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 3 10:30:10 2022] GET /identity/v3/groups/ebeaaf50bf9441d7992937759eff8370 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:11.021599 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37736be7-226c-4a3d-b2d9-6323b12a7db8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:11.034808 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 212/435] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:11.036604 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9da6a-3855-485b-9383-0ec20d144661 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:11.044839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:11.175788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37736be7-226c-4a3d-b2d9-6323b12a7db8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:11.221075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9da6a-3855-485b-9383-0ec20d144661 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:11.221416 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f7ca4-b61d-434f-b5b1-2441b8a2ab69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:11.223123 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f7ca4-b61d-434f-b5b1-2441b8a2ab69 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:11.223626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f7ca4-b61d-434f-b5b1-2441b8a2ab69 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:11.224121 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f7ca4-b61d-434f-b5b1-2441b8a2ab69 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:11.225538 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f7ca4-b61d-434f-b5b1-2441b8a2ab69 None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:11.226080 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f7ca4-b61d-434f-b5b1-2441b8a2ab69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:11.228274 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f7ca4-b61d-434f-b5b1-2441b8a2ab69 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:11.249983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 224/436] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:30:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:11.251948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f7ca4-b61d-434f-b5b1-2441b8a2ab69 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:11.267769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c678a9-1e23-483e-b07d-48540a3bb896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:11.269513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c678a9-1e23-483e-b07d-48540a3bb896 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:11.269830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c678a9-1e23-483e-b07d-48540a3bb896 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:11.270070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c678a9-1e23-483e-b07d-48540a3bb896 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:11.270867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c678a9-1e23-483e-b07d-48540a3bb896 None admin] RBAC: Adding query filter params (group.id=ebeaaf50bf9441d7992937759eff8370, role.id=82be5cfedcb3407f8d14522ef87247ac, scope.project.id=998a52ff5527415da541925c79fddaab) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:11.271178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c678a9-1e23-483e-b07d-48540a3bb896 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:11.273072 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c678a9-1e23-483e-b07d-48540a3bb896 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:11.280534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 213/437] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 3 10:30:11 2022] GET /identity/v3/role_assignments?group.id=ebeaaf50bf9441d7992937759eff8370&scope.project.id=998a52ff5527415da541925c79fddaab&role.id=82be5cfedcb3407f8d14522ef87247ac => generated 611 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:11.281087 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c678a9-1e23-483e-b07d-48540a3bb896 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:11.369636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:11.383741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:11.422122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f7ca4-b61d-434f-b5b1-2441b8a2ab69 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:11.435850 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c678a9-1e23-483e-b07d-48540a3bb896 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:11.474325 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbe078-bea5-4171-b3a7-1e68378afa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:11.475001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbe078-bea5-4171-b3a7-1e68378afa21 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:11.475586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbe078-bea5-4171-b3a7-1e68378afa21 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:11.475892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbe078-bea5-4171-b3a7-1e68378afa21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:11.486910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd00b6-e34d-46c1-8a26-bf58057a9534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:11.488402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd00b6-e34d-46c1-8a26-bf58057a9534 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:11.488852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd00b6-e34d-46c1-8a26-bf58057a9534 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:11.489281 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd00b6-e34d-46c1-8a26-bf58057a9534 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:11.490952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbe078-bea5-4171-b3a7-1e68378afa21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-e4dbe078-bea5-4171-b3a7-1e68378afa21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:11.498618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:11.499712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:11.500646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:11.500646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:11.500646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:11.500646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:11.500646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:11.500646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:11.500646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:11.500646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:11.500646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:11.500646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:11.500646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:30:11.500646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:11.505482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 225/438] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:30:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:11.507250 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbe078-bea5-4171-b3a7-1e68378afa21 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:11.608089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd00b6-e34d-46c1-8a26-bf58057a9534 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:11.609631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd00b6-e34d-46c1-8a26-bf58057a9534 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:11.663682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbe078-bea5-4171-b3a7-1e68378afa21 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:11.710822 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:11.727735 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f932ba-17ee-406e-95e2-71844a4f1302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:11.728536 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f932ba-17ee-406e-95e2-71844a4f1302 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:11.728992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f932ba-17ee-406e-95e2-71844a4f1302 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:11.729410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f932ba-17ee-406e-95e2-71844a4f1302 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:11.731046 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f932ba-17ee-406e-95e2-71844a4f1302 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:11.731557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f932ba-17ee-406e-95e2-71844a4f1302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:11.733678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f932ba-17ee-406e-95e2-71844a4f1302 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:11.737424 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:11.753539 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 226/439] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:11.753539 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777070d2-6877-440e-924c-8e22707e13c5 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:11.753539 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777070d2-6877-440e-924c-8e22707e13c5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:11.753539 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777070d2-6877-440e-924c-8e22707e13c5 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:11.755837 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 227/440] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:11.757065 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777070d2-6877-440e-924c-8e22707e13c5 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:11.777874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 214/441] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:11.780464 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d6c1d-03c4-4ac5-bba9-18e277e8aa23 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:11.781235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d6c1d-03c4-4ac5-bba9-18e277e8aa23 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:11.781773 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d6c1d-03c4-4ac5-bba9-18e277e8aa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:11.805702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ff2d6c1d-03c4-4ac5-bba9-18e277e8aa23 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:11.807234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ff2d6c1d-03c4-4ac5-bba9-18e277e8aa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:11.902950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 215/442] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:11.903863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d6c1d-03c4-4ac5-bba9-18e277e8aa23 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:11.925206 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777070d2-6877-440e-924c-8e22707e13c5 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:11.989725 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc06e3f-f828-4af8-a997-914cace57dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:11.990374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc06e3f-f828-4af8-a997-914cace57dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:11.990548 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc06e3f-f828-4af8-a997-914cace57dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:11.990826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc06e3f-f828-4af8-a997-914cace57dd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:12.028521 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:12.045185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d6c1d-03c4-4ac5-bba9-18e277e8aa23 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:12.070931 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:12.075431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc06e3f-f828-4af8-a997-914cace57dd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:12.077077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc06e3f-f828-4af8-a997-914cace57dd6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:12.086105 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e48efc-dbc0-4b2a-ba9c-8718b4a2afdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:12.086828 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e48efc-dbc0-4b2a-ba9c-8718b4a2afdb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:12.086828 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e48efc-dbc0-4b2a-ba9c-8718b4a2afdb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:12.087523 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e48efc-dbc0-4b2a-ba9c-8718b4a2afdb None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:12.088810 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e48efc-dbc0-4b2a-ba9c-8718b4a2afdb None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:12.090284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e48efc-dbc0-4b2a-ba9c-8718b4a2afdb None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:12.091966 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 216/443] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:30:11 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:12.092751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e48efc-dbc0-4b2a-ba9c-8718b4a2afdb None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:12.190939 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 228/444] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:12.192334 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc06e3f-f828-4af8-a997-914cace57dd6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:12.219965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e48efc-dbc0-4b2a-ba9c-8718b4a2afdb None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:12.260617 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc65da-9ff4-4d48-96b6-b6e65166e69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:12.261176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc65da-9ff4-4d48-96b6-b6e65166e69a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:12.261453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc65da-9ff4-4d48-96b6-b6e65166e69a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:12.261755 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc65da-9ff4-4d48-96b6-b6e65166e69a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:12.274992 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc65da-9ff4-4d48-96b6-b6e65166e69a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-56dc65da-9ff4-4d48-96b6-b6e65166e69a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:12.280071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:12.281069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:12.281921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:12.281921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:12.281921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:12.281921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:12.281921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:12.281921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:12.281921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:12.281921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:12.281921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:12.281921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:12.281921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:30:12.281921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:12.281921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 217/445] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:30:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:12.282578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc65da-9ff4-4d48-96b6-b6e65166e69a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:12.305896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:12.323405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:12.340564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc06e3f-f828-4af8-a997-914cace57dd6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:12.393820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec63c3c-dcf3-4b8d-b7ce-39922608ef90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:12.394561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec63c3c-dcf3-4b8d-b7ce-39922608ef90 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:12.394854 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec63c3c-dcf3-4b8d-b7ce-39922608ef90 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:12.395136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec63c3c-dcf3-4b8d-b7ce-39922608ef90 None admin] PATH_INFO: `/v3/groups/ebeaaf50bf9441d7992937759eff8370` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:12.396178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc65da-9ff4-4d48-96b6-b6e65166e69a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:12.396658 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec63c3c-dcf3-4b8d-b7ce-39922608ef90 None admin] RBAC: Authorizing `identity:get_group(group_id=ebeaaf50bf9441d7992937759eff8370)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:12.397834 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec63c3c-dcf3-4b8d-b7ce-39922608ef90 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:12.399469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 229/446] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 3 10:30:12 2022] GET /identity/v3/groups/ebeaaf50bf9441d7992937759eff8370 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:12.400251 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec63c3c-dcf3-4b8d-b7ce-39922608ef90 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:12.443606 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57746d70-17c1-470d-9550-8582a5e3b75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:12.446100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57746d70-17c1-470d-9550-8582a5e3b75e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:12.446100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57746d70-17c1-470d-9550-8582a5e3b75e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:12.446100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57746d70-17c1-470d-9550-8582a5e3b75e None admin] PATH_INFO: `/v3/projects/a90579cdd4284a47979efcd0ca29e39a` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:12.447135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57746d70-17c1-470d-9550-8582a5e3b75e None admin] RBAC: Authorizing `identity:get_project(project_id=a90579cdd4284a47979efcd0ca29e39a)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:12.447607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57746d70-17c1-470d-9550-8582a5e3b75e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:12.449498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 218/447] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:30:12 2022] GET /identity/v3/projects/a90579cdd4284a47979efcd0ca29e39a => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:12.450109 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57746d70-17c1-470d-9550-8582a5e3b75e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:12.536354 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec63c3c-dcf3-4b8d-b7ce-39922608ef90 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:12.543470 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:12.568855 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57746d70-17c1-470d-9550-8582a5e3b75e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:12.572736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5717d-3592-42e4-b3d3-442f41b77c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:12.573360 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5717d-3592-42e4-b3d3-442f41b77c0e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:12.573673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5717d-3592-42e4-b3d3-442f41b77c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:12.573947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5717d-3592-42e4-b3d3-442f41b77c0e None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:12.574666 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5717d-3592-42e4-b3d3-442f41b77c0e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:12.574966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5717d-3592-42e4-b3d3-442f41b77c0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:12.576341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5717d-3592-42e4-b3d3-442f41b77c0e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:12.584778 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 230/448] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:12.585661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5717d-3592-42e4-b3d3-442f41b77c0e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:12.618002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:12.618240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4334b41-c9df-41c5-be5d-0bd4fd59b8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:12.618638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4334b41-c9df-41c5-be5d-0bd4fd59b8b1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:12.618919 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4334b41-c9df-41c5-be5d-0bd4fd59b8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:12.619193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4334b41-c9df-41c5-be5d-0bd4fd59b8b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:12.620114 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4334b41-c9df-41c5-be5d-0bd4fd59b8b1 None admin] RBAC: Adding query filter params (group.id=ebeaaf50bf9441d7992937759eff8370, role.id=82be5cfedcb3407f8d14522ef87247ac, scope.project.id=d4db61efd90f48a6b63ae8fd15240329) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:12.620574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4334b41-c9df-41c5-be5d-0bd4fd59b8b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:12.622049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4334b41-c9df-41c5-be5d-0bd4fd59b8b1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:12.628246 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 219/449] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 3 10:30:12 2022] GET /identity/v3/role_assignments?group.id=ebeaaf50bf9441d7992937759eff8370&scope.project.id=d4db61efd90f48a6b63ae8fd15240329&role.id=82be5cfedcb3407f8d14522ef87247ac => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:12.629079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4334b41-c9df-41c5-be5d-0bd4fd59b8b1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:12.720307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5717d-3592-42e4-b3d3-442f41b77c0e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:12.774461 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4334b41-c9df-41c5-be5d-0bd4fd59b8b1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:12.776844 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a182c21-118f-4642-a72b-9b77e1e58bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:12.777541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a182c21-118f-4642-a72b-9b77e1e58bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:12.777826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a182c21-118f-4642-a72b-9b77e1e58bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:12.778104 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a182c21-118f-4642-a72b-9b77e1e58bc6 None admin] PATH_INFO: `/v3/groups/ec0cfcbbb96344a89ff8119338799d86` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:12.779385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a182c21-118f-4642-a72b-9b77e1e58bc6 None admin] RBAC: Authorizing `identity:get_group(group_id=ec0cfcbbb96344a89ff8119338799d86)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:12.780871 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a182c21-118f-4642-a72b-9b77e1e58bc6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:12.782610 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 231/450] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 3 10:30:12 2022] GET /identity/v3/groups/ec0cfcbbb96344a89ff8119338799d86 => generated 229 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:12.790510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a182c21-118f-4642-a72b-9b77e1e58bc6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:12.825679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a20351-30b8-4521-876c-3661af2bc41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:12.826419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a20351-30b8-4521-876c-3661af2bc41c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:12.826916 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a20351-30b8-4521-876c-3661af2bc41c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:12.827361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a20351-30b8-4521-876c-3661af2bc41c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:12.850895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:12.927120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:12.928863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a20351-30b8-4521-876c-3661af2bc41c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:12.930394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a20351-30b8-4521-876c-3661af2bc41c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:12.937244 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a182c21-118f-4642-a72b-9b77e1e58bc6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:12.993909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2355e68-f0e2-4a87-9ad9-87070dd88956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:12.994808 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2355e68-f0e2-4a87-9ad9-87070dd88956 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:12.994960 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2355e68-f0e2-4a87-9ad9-87070dd88956 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:12.995106 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2355e68-f0e2-4a87-9ad9-87070dd88956 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:12.995927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2355e68-f0e2-4a87-9ad9-87070dd88956 None admin] RBAC: Adding query filter params (group.id=ec0cfcbbb96344a89ff8119338799d86, role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=a90579cdd4284a47979efcd0ca29e39a) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:12.996238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2355e68-f0e2-4a87-9ad9-87070dd88956 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:12.998744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2355e68-f0e2-4a87-9ad9-87070dd88956 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:13.007313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 232/451] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 3 10:30:12 2022] GET /identity/v3/role_assignments?group.id=ec0cfcbbb96344a89ff8119338799d86&scope.project.id=a90579cdd4284a47979efcd0ca29e39a&role.id=01714797cab143e1a7a150c068fa9315 => generated 611 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:30:13.090878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 220/452] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:13.097157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2355e68-f0e2-4a87-9ad9-87070dd88956 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:13.221316 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2355e68-f0e2-4a87-9ad9-87070dd88956 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:13.262074 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3374c21-5f5c-43c7-aae7-0287edcef8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:13.262723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3374c21-5f5c-43c7-aae7-0287edcef8cc None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:13.263002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3374c21-5f5c-43c7-aae7-0287edcef8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:13.263270 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3374c21-5f5c-43c7-aae7-0287edcef8cc None admin] PATH_INFO: `/v3/projects/d4db61efd90f48a6b63ae8fd15240329` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:13.264459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3374c21-5f5c-43c7-aae7-0287edcef8cc None admin] RBAC: Authorizing `identity:get_project(project_id=d4db61efd90f48a6b63ae8fd15240329)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:13.265949 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3374c21-5f5c-43c7-aae7-0287edcef8cc None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:13.267619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 233/453] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:30:13 2022] GET /identity/v3/projects/d4db61efd90f48a6b63ae8fd15240329 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:13.274219 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a20351-30b8-4521-876c-3661af2bc41c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:13.325095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:13.381874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a20351-30b8-4521-876c-3661af2bc41c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:13.417292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73893c48-bc3c-43be-8113-99fafc350194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:13.418014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73893c48-bc3c-43be-8113-99fafc350194 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:13.418252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73893c48-bc3c-43be-8113-99fafc350194 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:13.418529 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73893c48-bc3c-43be-8113-99fafc350194 None admin] PATH_INFO: `/v3/groups/ebeaaf50bf9441d7992937759eff8370` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:13.419712 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73893c48-bc3c-43be-8113-99fafc350194 None admin] RBAC: Authorizing `identity:get_group(group_id=ebeaaf50bf9441d7992937759eff8370)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:13.421146 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73893c48-bc3c-43be-8113-99fafc350194 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:13.422837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 221/454] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 3 10:30:13 2022] GET /identity/v3/groups/ebeaaf50bf9441d7992937759eff8370 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:13.429638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3374c21-5f5c-43c7-aae7-0287edcef8cc None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:13.463233 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:13.540471 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3374c21-5f5c-43c7-aae7-0287edcef8cc None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:13.577830 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97933758-f119-4636-9a75-42e90c4a72bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:13.578452 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97933758-f119-4636-9a75-42e90c4a72bd None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:13.578734 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97933758-f119-4636-9a75-42e90c4a72bd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:13.579007 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97933758-f119-4636-9a75-42e90c4a72bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:13.579804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97933758-f119-4636-9a75-42e90c4a72bd None admin] RBAC: Adding query filter params (group.id=ebeaaf50bf9441d7992937759eff8370, role.id=cde4978cbb9648639ff720b5ca82886b, scope.project.id=d4db61efd90f48a6b63ae8fd15240329) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:13.580114 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97933758-f119-4636-9a75-42e90c4a72bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:13.581615 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97933758-f119-4636-9a75-42e90c4a72bd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:13.588038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 234/455] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 3 10:30:13 2022] GET /identity/v3/role_assignments?group.id=ebeaaf50bf9441d7992937759eff8370&scope.project.id=d4db61efd90f48a6b63ae8fd15240329&role.id=cde4978cbb9648639ff720b5ca82886b => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:17.008929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d6b91-c705-43d0-b399-55787d42cb4c None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:17.009500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d6b91-c705-43d0-b399-55787d42cb4c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:17.010207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d6b91-c705-43d0-b399-55787d42cb4c None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:17.012939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 222/456] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:17.022029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa567d1d-db8e-4535-81f9-77a687fc1108 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:17.022552 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa567d1d-db8e-4535-81f9-77a687fc1108 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:17.022934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa567d1d-db8e-4535-81f9-77a687fc1108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:17.083302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-aa567d1d-db8e-4535-81f9-77a687fc1108 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:17.085392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-aa567d1d-db8e-4535-81f9-77a687fc1108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:17.296311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 235/457] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:30:17.322666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb7264-b44c-44b1-a5c1-e2837266aef6 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:17.323320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb7264-b44c-44b1-a5c1-e2837266aef6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:17.323800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb7264-b44c-44b1-a5c1-e2837266aef6 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:17.325826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 223/458] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:17.332970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874c922-84be-4a40-a0fd-2d72f49792a6 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:17.333508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874c922-84be-4a40-a0fd-2d72f49792a6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:17.334001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874c922-84be-4a40-a0fd-2d72f49792a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:17.342849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb7264-b44c-44b1-a5c1-e2837266aef6 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:17.356495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6874c922-84be-4a40-a0fd-2d72f49792a6 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:17.357743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6874c922-84be-4a40-a0fd-2d72f49792a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:17.393338 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:17.533704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 236/459] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:17.546045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb7264-b44c-44b1-a5c1-e2837266aef6 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:17.579454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:17.608010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b115c58-c99e-423a-9d5a-80ae19a53c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:17.613813 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b115c58-c99e-423a-9d5a-80ae19a53c09 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:17.614934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b115c58-c99e-423a-9d5a-80ae19a53c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:17.615427 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b115c58-c99e-423a-9d5a-80ae19a53c09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:17.622188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874c922-84be-4a40-a0fd-2d72f49792a6 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:17.647387 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b115c58-c99e-423a-9d5a-80ae19a53c09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:17.651014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b115c58-c99e-423a-9d5a-80ae19a53c09 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:17.693846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-7b115c58-c99e-423a-9d5a-80ae19a53c09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:17.693846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:17.693846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:17.693846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:17.693846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:17.693846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:17.693846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:17.693846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:17.693846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:17.693846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:17.693846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:17.693846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:17.693846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:17.693846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:17.693846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:17.695382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:17.696933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:17.696933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:17.696933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:17.696933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:17.698006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 224/460] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:17.710548 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b115c58-c99e-423a-9d5a-80ae19a53c09 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:17.804822 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874c922-84be-4a40-a0fd-2d72f49792a6 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:17.860544 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b115c58-c99e-423a-9d5a-80ae19a53c09 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:17.860954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92461f9a-8903-4a04-b51a-ca9d1d1e55d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:17.865519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92461f9a-8903-4a04-b51a-ca9d1d1e55d3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:17.865519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92461f9a-8903-4a04-b51a-ca9d1d1e55d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:17.865519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92461f9a-8903-4a04-b51a-ca9d1d1e55d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:17.877011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92461f9a-8903-4a04-b51a-ca9d1d1e55d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:17.879813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92461f9a-8903-4a04-b51a-ca9d1d1e55d3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:17.901987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-92461f9a-8903-4a04-b51a-ca9d1d1e55d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:17.901987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:17.901987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:17.901987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:17.901987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:17.901987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:17.901987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:17.901987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:17.901987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:17.901987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:17.901987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:17.901987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:17.901987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:17.901987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:17.901987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:17.902888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:17.903726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:17.903726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:17.903726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:17.903726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:17.905028 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2414cc-0f8d-4cc0-985b-ae2dc050f7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:17.905701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2414cc-0f8d-4cc0-985b-ae2dc050f7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:17.905983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2414cc-0f8d-4cc0-985b-ae2dc050f7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:17.906376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2414cc-0f8d-4cc0-985b-ae2dc050f7d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:17.906796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 237/461] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:30:17.907041 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2414cc-0f8d-4cc0-985b-ae2dc050f7d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:17.907476 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2414cc-0f8d-4cc0-985b-ae2dc050f7d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:17.908721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a21d04-c453-4439-b46d-8b2ecf6313a3 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:17.908952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2414cc-0f8d-4cc0-985b-ae2dc050f7d7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:17.909430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a21d04-c453-4439-b46d-8b2ecf6313a3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:17.909992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a21d04-c453-4439-b46d-8b2ecf6313a3 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:17.916012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 238/462] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:17.916012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a21d04-c453-4439-b46d-8b2ecf6313a3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:17.921827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:17.936484 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 225/463] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:30:17.937894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007c89e-5ed3-46b6-b374-6fe738f1cd8e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:17.938182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007c89e-5ed3-46b6-b374-6fe738f1cd8e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:17.938457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007c89e-5ed3-46b6-b374-6fe738f1cd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:17.964229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c007c89e-5ed3-46b6-b374-6fe738f1cd8e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:17.965634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c007c89e-5ed3-46b6-b374-6fe738f1cd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:17.996552 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:18.050551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 226/464] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:18.050551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007c89e-5ed3-46b6-b374-6fe738f1cd8e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:18.079677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a21d04-c453-4439-b46d-8b2ecf6313a3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:18.120758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550eae52-c574-44df-abd0-b8776c80a6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:18.120956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550eae52-c574-44df-abd0-b8776c80a6cb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:18.121354 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550eae52-c574-44df-abd0-b8776c80a6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:18.121831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550eae52-c574-44df-abd0-b8776c80a6cb None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:18.122644 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550eae52-c574-44df-abd0-b8776c80a6cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:18.123052 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550eae52-c574-44df-abd0-b8776c80a6cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:18.124543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550eae52-c574-44df-abd0-b8776c80a6cb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:18.139514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 239/465] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:18.140582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550eae52-c574-44df-abd0-b8776c80a6cb None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:18.210332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007c89e-5ed3-46b6-b374-6fe738f1cd8e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:18.221579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:18.268518 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb9e33-ebfa-4c52-b677-149f3d026c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:18.269894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb9e33-ebfa-4c52-b677-149f3d026c91 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:18.270918 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb9e33-ebfa-4c52-b677-149f3d026c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:18.271667 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb9e33-ebfa-4c52-b677-149f3d026c91 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:18.272596 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb9e33-ebfa-4c52-b677-149f3d026c91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:18.273278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550eae52-c574-44df-abd0-b8776c80a6cb None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:18.274527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb9e33-ebfa-4c52-b677-149f3d026c91 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:18.312260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 227/466] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:30:18 2022] POST /identity/v3/users => generated 246 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:30:18.313638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb9e33-ebfa-4c52-b677-149f3d026c91 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:18.335112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fb041-73ca-493d-a604-fd6431053b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:18.336152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fb041-73ca-493d-a604-fd6431053b65 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:18.336434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fb041-73ca-493d-a604-fd6431053b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:18.336805 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fb041-73ca-493d-a604-fd6431053b65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:18.344107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:18.355179 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fb041-73ca-493d-a604-fd6431053b65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:18.357444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fb041-73ca-493d-a604-fd6431053b65 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:18.382772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-0e2fb041-73ca-493d-a604-fd6431053b65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:18.382772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:18.382772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:18.382772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:18.382772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:18.382772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:18.382772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:18.382772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:18.382772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:18.382772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:18.382772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:18.382772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:18.382772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:18.382772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:18.382772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:18.383704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:18.384626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:18.384626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:18.384626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:18.384626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:18.393033 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 240/467] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:18.398146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984402ce-c577-45fd-b752-b8a394b2ca8a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:18.398732 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984402ce-c577-45fd-b752-b8a394b2ca8a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:18.399018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984402ce-c577-45fd-b752-b8a394b2ca8a None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:18.400623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 241/468] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:18.403444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66a70d-94f1-4ddf-9bb8-6a4cabb7abd7 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:18.403754 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66a70d-94f1-4ddf-9bb8-6a4cabb7abd7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:18.403991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66a70d-94f1-4ddf-9bb8-6a4cabb7abd7 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:18.405744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 242/469] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:18.406896 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66a70d-94f1-4ddf-9bb8-6a4cabb7abd7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:18.486755 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb9e33-ebfa-4c52-b677-149f3d026c91 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:18.531011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545cd676-8bd0-4969-8abe-edd5c925f6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:18.532185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cd676-8bd0-4969-8abe-edd5c925f6ea None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:18.532481 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cd676-8bd0-4969-8abe-edd5c925f6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:18.532768 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cd676-8bd0-4969-8abe-edd5c925f6ea None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:18.533837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545cd676-8bd0-4969-8abe-edd5c925f6ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:18.535921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545cd676-8bd0-4969-8abe-edd5c925f6ea None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:18.563338 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66a70d-94f1-4ddf-9bb8-6a4cabb7abd7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:18.564317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 228/470] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:30:18 2022] POST /identity/v3/users => generated 248 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:30:18.565820 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f1225-fe50-441c-a06a-ea2f5a97caaf None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:18.566158 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f1225-fe50-441c-a06a-ea2f5a97caaf None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:18.566393 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f1225-fe50-441c-a06a-ea2f5a97caaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:18.586979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-504f1225-fe50-441c-a06a-ea2f5a97caaf None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:18.587686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:18.588416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-504f1225-fe50-441c-a06a-ea2f5a97caaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:18.637210 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690477f2-0aa6-4be4-b56f-dc48801c79d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:18.637815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690477f2-0aa6-4be4-b56f-dc48801c79d3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:18.638125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690477f2-0aa6-4be4-b56f-dc48801c79d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:18.638406 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690477f2-0aa6-4be4-b56f-dc48801c79d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:18.639153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690477f2-0aa6-4be4-b56f-dc48801c79d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:18.639450 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690477f2-0aa6-4be4-b56f-dc48801c79d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:18.640895 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690477f2-0aa6-4be4-b56f-dc48801c79d3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:18.664831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 243/471] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:18.667126 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a8f8f-2007-43be-b5b0-8711a6878b71 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:18.667589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a8f8f-2007-43be-b5b0-8711a6878b71 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:18.668059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a8f8f-2007-43be-b5b0-8711a6878b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:18.699296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d65a8f8f-2007-43be-b5b0-8711a6878b71 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:18.701385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d65a8f8f-2007-43be-b5b0-8711a6878b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:18.734991 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 229/472] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:18.736991 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f1225-fe50-441c-a06a-ea2f5a97caaf None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:18.741704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:18.812613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 244/473] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:18.812613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a8f8f-2007-43be-b5b0-8711a6878b71 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:18.934095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f1225-fe50-441c-a06a-ea2f5a97caaf None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:18.969516 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a8f8f-2007-43be-b5b0-8711a6878b71 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:18.972830 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:19.009853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa7110-d522-4fa0-93a0-bdac9d4fc98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:19.010492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa7110-d522-4fa0-93a0-bdac9d4fc98a None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:19.010771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa7110-d522-4fa0-93a0-bdac9d4fc98a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:19.011044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa7110-d522-4fa0-93a0-bdac9d4fc98a None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:19.011774 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa7110-d522-4fa0-93a0-bdac9d4fc98a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:19.013154 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa7110-d522-4fa0-93a0-bdac9d4fc98a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:19.015392 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5b362-47c4-4666-b29e-f8ec0869ed6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:19.016559 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5b362-47c4-4666-b29e-f8ec0869ed6d None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:19.017143 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5b362-47c4-4666-b29e-f8ec0869ed6d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:19.017772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5b362-47c4-4666-b29e-f8ec0869ed6d None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:19.019554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5b362-47c4-4666-b29e-f8ec0869ed6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:19.021718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.sql.core [None req-04aa7110-d522-4fa0-93a0-bdac9d4fc98a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 03 10:30:19.021718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 03 10:30:19.021718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [parameters: {'id': 'f5a1b199424c46e189c10430cb327b38', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 03 10:30:19.021718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80318) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 03 10:30:19.022303 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5b362-47c4-4666-b29e-f8ec0869ed6d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:19.042972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-04aa7110-d522-4fa0-93a0-bdac9d4fc98a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 03 10:30:19.042972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:19.042972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 10:30:19.042972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 10:30:19.042972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 10:30:19.042972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 10:30:19.042972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 10:30:19.042972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 10:30:19.042972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 10:30:19.042972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application conn.query(q) Aug 03 10:30:19.042972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 10:30:19.042972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 10:30:19.042972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 10:30:19.042972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result.read() Aug 03 10:30:19.042972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return ref.to_dict() Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 10:30:19.044385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application next(self.gen) Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application yield resource Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application next(self.gen) Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.session.rollback() Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.force_reraise() Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise self.value Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application session.commit() Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 03 10:30:19.046040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._prepare_impl() Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.session.flush() Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._flush(objects) Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application compat.raise_( Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application flush_context.execute() Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rec.execute(self) Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 03 10:30:19.047962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 10:30:19.059970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application conn.query(q) Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result.read() Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 10:30:19.067283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [parameters: {'id': 'f5a1b199424c46e189c10430cb327b38', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:19.068824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 03 10:30:19.070489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:19.072029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 245/474] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Aug 3 10:30:18 2022] POST /identity/v3/roles => generated 144 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:19.072029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa7110-d522-4fa0-93a0-bdac9d4fc98a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:19.072258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 230/475] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:30:18 2022] POST /identity/v3/users => generated 248 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:30:19.072258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5b362-47c4-4666-b29e-f8ec0869ed6d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:19.292771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:19.347853 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5b362-47c4-4666-b29e-f8ec0869ed6d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:19.374723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa7110-d522-4fa0-93a0-bdac9d4fc98a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:19.392135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9b344-0d6b-4108-a5f3-7bc86d6d91c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:19.392424 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9b344-0d6b-4108-a5f3-7bc86d6d91c1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:19.392555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9b344-0d6b-4108-a5f3-7bc86d6d91c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:19.394046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9b344-0d6b-4108-a5f3-7bc86d6d91c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:19.419647 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9b344-0d6b-4108-a5f3-7bc86d6d91c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-44b9b344-0d6b-4108-a5f3-7bc86d6d91c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:19.426089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:19.427782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:19.429252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:19.429252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:19.429252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:19.429252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:19.429252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:19.429252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:19.429252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:19.429252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:19.429252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:19.429252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:19.429252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 10:30:19.429252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:19.429252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 231/476] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:30:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:19.441061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9b344-0d6b-4108-a5f3-7bc86d6d91c1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:19.470513 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e9f4f-c9c7-4951-8c27-6ea7f6873eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:19.472430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e9f4f-c9c7-4951-8c27-6ea7f6873eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:19.472910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e9f4f-c9c7-4951-8c27-6ea7f6873eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:19.473286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e9f4f-c9c7-4951-8c27-6ea7f6873eb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:19.485433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e9f4f-c9c7-4951-8c27-6ea7f6873eb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:19.487846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e9f4f-c9c7-4951-8c27-6ea7f6873eb5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:19.509038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-8a8e9f4f-c9c7-4951-8c27-6ea7f6873eb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:19.509038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:19.509038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:19.509038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:19.509038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:19.509038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:19.509038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:19.509038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:19.509038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:19.509038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:19.509038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:19.509038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:19.509038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:19.509038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:19.509038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:19.510893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:19.512752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:19.512752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:19.512752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:19.512752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:19.512752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 246/477] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 462 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:19.519453 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e9f4f-c9c7-4951-8c27-6ea7f6873eb5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:19.551542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:19.651304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e9f4f-c9c7-4951-8c27-6ea7f6873eb5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:19.694383 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d307e75-2324-4cee-ae55-5812f65a2d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:19.695151 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d307e75-2324-4cee-ae55-5812f65a2d45 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:19.695543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d307e75-2324-4cee-ae55-5812f65a2d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:19.695918 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d307e75-2324-4cee-ae55-5812f65a2d45 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:19.696746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d307e75-2324-4cee-ae55-5812f65a2d45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:19.697152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d307e75-2324-4cee-ae55-5812f65a2d45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:19.698764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d307e75-2324-4cee-ae55-5812f65a2d45 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:19.715784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 247/478] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:19.721532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d307e75-2324-4cee-ae55-5812f65a2d45 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:19.761496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9b344-0d6b-4108-a5f3-7bc86d6d91c1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:19.783183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:19.814319 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf7ca8-a719-40f7-bef0-9afe88d46718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:19.815208 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf7ca8-a719-40f7-bef0-9afe88d46718 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:19.816105 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf7ca8-a719-40f7-bef0-9afe88d46718 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:19.816487 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf7ca8-a719-40f7-bef0-9afe88d46718 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:19.817370 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf7ca8-a719-40f7-bef0-9afe88d46718 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:19.817870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf7ca8-a719-40f7-bef0-9afe88d46718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:19.819416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf7ca8-a719-40f7-bef0-9afe88d46718 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:19.849826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 232/479] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:30:19.907554 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:19.944133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d307e75-2324-4cee-ae55-5812f65a2d45 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:20.010471 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946d4c9-6de0-4cab-abaa-d0ee1dee52fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:20.011429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946d4c9-6de0-4cab-abaa-d0ee1dee52fb None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:20.011859 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946d4c9-6de0-4cab-abaa-d0ee1dee52fb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:20.012282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946d4c9-6de0-4cab-abaa-d0ee1dee52fb None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:20.013399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946d4c9-6de0-4cab-abaa-d0ee1dee52fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:20.015847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946d4c9-6de0-4cab-abaa-d0ee1dee52fb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:20.055263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 248/480] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:30:19 2022] POST /identity/v3/users => generated 249 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:30:22.520469 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e7e6f-a7d4-4279-bb8d-faa39d0b1a43 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:22.520469 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e7e6f-a7d4-4279-bb8d-faa39d0b1a43 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:22.520469 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e7e6f-a7d4-4279-bb8d-faa39d0b1a43 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:22.520469 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 233/481] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:30:22.529686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d497911-b924-450c-8875-2c578f23e9c5 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:22.530342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d497911-b924-450c-8875-2c578f23e9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:22.530766 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d497911-b924-450c-8875-2c578f23e9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:22.543263 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9bb7e-2980-4d04-94ec-fa682cae4cc2 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:22.543876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9bb7e-2980-4d04-94ec-fa682cae4cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:22.544375 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9bb7e-2980-4d04-94ec-fa682cae4cc2 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:22.545980 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 234/482] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:30:22.556239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20eff9-c821-4ece-9a97-cbfea07fd75a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:22.556712 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20eff9-c821-4ece-9a97-cbfea07fd75a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:22.557172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20eff9-c821-4ece-9a97-cbfea07fd75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:22.571033 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6d497911-b924-450c-8875-2c578f23e9c5 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:22.572675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6d497911-b924-450c-8875-2c578f23e9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:22.655374 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ce20eff9-c821-4ece-9a97-cbfea07fd75a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:22.665589 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ce20eff9-c821-4ece-9a97-cbfea07fd75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:22.816807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 249/483] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:22.819536 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fa593-13f7-4961-bce1-d4b493bcbdd1 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:22.820282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fa593-13f7-4961-bce1-d4b493bcbdd1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:22.821119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fa593-13f7-4961-bce1-d4b493bcbdd1 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:22.829727 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 250/484] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:22.834530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779aff2-69e1-4224-8a5a-e95f5c552530 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:22.835281 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779aff2-69e1-4224-8a5a-e95f5c552530 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:22.835958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779aff2-69e1-4224-8a5a-e95f5c552530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:22.861126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 235/485] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:30:22.863032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20eff9-c821-4ece-9a97-cbfea07fd75a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:22.885272 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9779aff2-69e1-4224-8a5a-e95f5c552530 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:22.887039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9779aff2-69e1-4224-8a5a-e95f5c552530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:23.015885 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20eff9-c821-4ece-9a97-cbfea07fd75a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:23.030650 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:23.032385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:23.043883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 251/486] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:30:23.045650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779aff2-69e1-4224-8a5a-e95f5c552530 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:23.070797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70cc39-34ee-4d4f-af9f-4a4e9d4e8cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:23.071925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70cc39-34ee-4d4f-af9f-4a4e9d4e8cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:23.072441 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70cc39-34ee-4d4f-af9f-4a4e9d4e8cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:23.072913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70cc39-34ee-4d4f-af9f-4a4e9d4e8cc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:23.086730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70cc39-34ee-4d4f-af9f-4a4e9d4e8cc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-8f70cc39-34ee-4d4f-af9f-4a4e9d4e8cc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:23.092686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:23.093706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:23.094499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:23.094499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:23.094499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:23.094499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:23.094499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:23.094499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:23.094499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:23.094499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:23.094499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:23.094499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:23.094499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:23.094499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:23.102237 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 236/487] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:23.103985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70cc39-34ee-4d4f-af9f-4a4e9d4e8cc6 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:23.223778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70cc39-34ee-4d4f-af9f-4a4e9d4e8cc6 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:23.252377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779aff2-69e1-4224-8a5a-e95f5c552530 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:23.269006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937a257-7b86-4c67-97fb-37d2c40d63ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:23.270082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937a257-7b86-4c67-97fb-37d2c40d63ee None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:23.270653 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937a257-7b86-4c67-97fb-37d2c40d63ee None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:23.271149 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937a257-7b86-4c67-97fb-37d2c40d63ee None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:23.272089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937a257-7b86-4c67-97fb-37d2c40d63ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:23.272613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937a257-7b86-4c67-97fb-37d2c40d63ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:23.274411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937a257-7b86-4c67-97fb-37d2c40d63ee None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:23.289067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 237/488] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:23.290751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937a257-7b86-4c67-97fb-37d2c40d63ee None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:23.314754 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff75139-7fae-40d0-bf9a-69cebe2a3e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:23.316353 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff75139-7fae-40d0-bf9a-69cebe2a3e74 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:23.316796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff75139-7fae-40d0-bf9a-69cebe2a3e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:23.317274 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff75139-7fae-40d0-bf9a-69cebe2a3e74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:23.332896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:23.345348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff75139-7fae-40d0-bf9a-69cebe2a3e74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-bff75139-7fae-40d0-bf9a-69cebe2a3e74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:23.355257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:23.356189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:23.356989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:23.356989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:23.356989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:23.356989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:23.356989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:23.356989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:23.356989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:23.356989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:23.356989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:23.356989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:23.356989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:23.356989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:23.364115 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 252/489] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:23.365774 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff75139-7fae-40d0-bf9a-69cebe2a3e74 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:23.420027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937a257-7b86-4c67-97fb-37d2c40d63ee None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:23.461607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2de2a6-0c9c-423d-b2e0-2ea3b743f8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:23.462323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2de2a6-0c9c-423d-b2e0-2ea3b743f8fb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:23.462886 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2de2a6-0c9c-423d-b2e0-2ea3b743f8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:23.462993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2de2a6-0c9c-423d-b2e0-2ea3b743f8fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:23.473609 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:23.479226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2de2a6-0c9c-423d-b2e0-2ea3b743f8fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-1f2de2a6-0c9c-423d-b2e0-2ea3b743f8fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:23.485185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:23.486150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:23.487531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:23.487531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:23.487531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:23.487531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:23.487531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:23.487531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:23.487531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:23.487531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:23.487531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:23.487531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:23.487531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:23.487531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:23.492413 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 238/490] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:23.494771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2de2a6-0c9c-423d-b2e0-2ea3b743f8fb None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:23.525626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff75139-7fae-40d0-bf9a-69cebe2a3e74 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:23.567573 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539af1c-9920-40e4-ba26-fce97c95cfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:23.569048 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539af1c-9920-40e4-ba26-fce97c95cfcc None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:23.569699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539af1c-9920-40e4-ba26-fce97c95cfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:23.570198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539af1c-9920-40e4-ba26-fce97c95cfcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:23.582608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539af1c-9920-40e4-ba26-fce97c95cfcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:23.584479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539af1c-9920-40e4-ba26-fce97c95cfcc None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:23.600852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-0539af1c-9920-40e4-ba26-fce97c95cfcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:23.600852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:23.600852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:23.600852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:23.600852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:23.600852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:23.600852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:23.600852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:23.600852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:23.600852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:23.600852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:23.600852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:23.600852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:23.600852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:23.600852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:23.601840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:23.602698 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:23.602698 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:23.602698 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:23.602698 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:23.604975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 253/491] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:23.606130 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539af1c-9920-40e4-ba26-fce97c95cfcc None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:23.613026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:23.626902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2de2a6-0c9c-423d-b2e0-2ea3b743f8fb None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:23.669177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8de49-69d3-437f-bd02-2e54c2632c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:23.670245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8de49-69d3-437f-bd02-2e54c2632c1d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:23.670725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8de49-69d3-437f-bd02-2e54c2632c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:23.671266 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8de49-69d3-437f-bd02-2e54c2632c1d None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:23.672332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8de49-69d3-437f-bd02-2e54c2632c1d None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:23.672863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8de49-69d3-437f-bd02-2e54c2632c1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:23.674641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8de49-69d3-437f-bd02-2e54c2632c1d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:23.685208 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 239/492] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:23.686606 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8de49-69d3-437f-bd02-2e54c2632c1d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:23.735893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539af1c-9920-40e4-ba26-fce97c95cfcc None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:23.747444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:23.780427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca4901-2903-48da-83a5-c92f4872fdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:23.781062 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca4901-2903-48da-83a5-c92f4872fdbd None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:23.781344 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca4901-2903-48da-83a5-c92f4872fdbd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:23.781656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca4901-2903-48da-83a5-c92f4872fdbd None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:23.782394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca4901-2903-48da-83a5-c92f4872fdbd None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:23.782700 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca4901-2903-48da-83a5-c92f4872fdbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:23.784213 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca4901-2903-48da-83a5-c92f4872fdbd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:23.794068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 254/493] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:23.794928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca4901-2903-48da-83a5-c92f4872fdbd None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:23.822436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8de49-69d3-437f-bd02-2e54c2632c1d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:23.862133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd32b3-59a8-4ab1-8716-33eae879c993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:23.862673 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd32b3-59a8-4ab1-8716-33eae879c993 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:23.862964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd32b3-59a8-4ab1-8716-33eae879c993 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:23.863244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd32b3-59a8-4ab1-8716-33eae879c993 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:23.863962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd32b3-59a8-4ab1-8716-33eae879c993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:23.864356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd32b3-59a8-4ab1-8716-33eae879c993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:23.865808 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd32b3-59a8-4ab1-8716-33eae879c993 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:23.882264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 240/494] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:23.883342 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ffe4c-d79e-4d22-9b02-7e4eef10df5c None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:23.883681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ffe4c-d79e-4d22-9b02-7e4eef10df5c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:23.883922 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ffe4c-d79e-4d22-9b02-7e4eef10df5c None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:23.885574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 241/495] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:23.890506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ffe4c-d79e-4d22-9b02-7e4eef10df5c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:23.911704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:23.937503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca4901-2903-48da-83a5-c92f4872fdbd None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:23.985008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cee907-251f-477d-a7a7-4acc840e553b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:23.985817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cee907-251f-477d-a7a7-4acc840e553b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:23.986176 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cee907-251f-477d-a7a7-4acc840e553b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:23.986539 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cee907-251f-477d-a7a7-4acc840e553b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:24.001363 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cee907-251f-477d-a7a7-4acc840e553b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:24.003546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cee907-251f-477d-a7a7-4acc840e553b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:24.021873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-24cee907-251f-477d-a7a7-4acc840e553b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:24.021873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:24.021873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:24.021873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:24.021873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:24.021873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:24.021873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:24.021873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:24.021873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:24.021873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:24.021873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:24.021873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:24.021873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:24.021873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:24.021873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:24.022792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:24.023624 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:24.023624 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:24.023624 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:24.023624 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:24.027771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 255/496] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:24.029070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cee907-251f-477d-a7a7-4acc840e553b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:24.045018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ffe4c-d79e-4d22-9b02-7e4eef10df5c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:24.059571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:24.089116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39818c41-8590-4e24-b723-db6b8e331bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:24.089814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39818c41-8590-4e24-b723-db6b8e331bea None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:24.090091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39818c41-8590-4e24-b723-db6b8e331bea None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:24.090364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39818c41-8590-4e24-b723-db6b8e331bea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:24.102784 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39818c41-8590-4e24-b723-db6b8e331bea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:24.104272 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39818c41-8590-4e24-b723-db6b8e331bea None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:24.122036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-39818c41-8590-4e24-b723-db6b8e331bea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:24.122036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:24.122036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:24.122036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:24.122036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:24.122036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:24.122036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:24.122036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:24.122036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:24.122036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:24.122036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:24.122036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:24.122036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:24.122036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:24.122036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:24.123654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:24.125401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:24.125401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:24.125401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:24.125401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:24.125401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 242/497] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:24.125401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e822b-e4ed-4e16-9041-587eba779708 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:24.125401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e822b-e4ed-4e16-9041-587eba779708 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:24.125401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e822b-e4ed-4e16-9041-587eba779708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:24.144485 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2c6e822b-e4ed-4e16-9041-587eba779708 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:24.145733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2c6e822b-e4ed-4e16-9041-587eba779708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:24.176032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cee907-251f-477d-a7a7-4acc840e553b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:24.220049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff5ea6-497a-4257-b606-5fd639d15c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:24.220704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff5ea6-497a-4257-b606-5fd639d15c14 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:24.220985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff5ea6-497a-4257-b606-5fd639d15c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:24.221264 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff5ea6-497a-4257-b606-5fd639d15c14 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:24.223939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:24.231022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff5ea6-497a-4257-b606-5fd639d15c14 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:24.232482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff5ea6-497a-4257-b606-5fd639d15c14 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:24.233331 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 243/498] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:24.234348 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e822b-e4ed-4e16-9041-587eba779708 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-a2ff5ea6-497a-4257-b606-5fd639d15c14 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:24.258743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:24.260572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:24.262388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:24.262388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:24.262388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:24.262388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:24.262388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:24.262388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:24.262388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:24.262388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:24.262388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 10:30:24.262388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:24.265960 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 256/499] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:24.267044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad006a-a054-4b00-9b3c-b94fade368a4 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:24.267319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad006a-a054-4b00-9b3c-b94fade368a4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:24.267592 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad006a-a054-4b00-9b3c-b94fade368a4 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:24.269509 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 257/500] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:24.270256 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad006a-a054-4b00-9b3c-b94fade368a4 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:24.368690 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e822b-e4ed-4e16-9041-587eba779708 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:24.382128 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:24.405662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad006a-a054-4b00-9b3c-b94fade368a4 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:24.418880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32163a4a-65f3-4325-9322-9eca3ad484e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:24.419762 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32163a4a-65f3-4325-9322-9eca3ad484e2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:24.420184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32163a4a-65f3-4325-9322-9eca3ad484e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:24.420540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32163a4a-65f3-4325-9322-9eca3ad484e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:24.421389 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32163a4a-65f3-4325-9322-9eca3ad484e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:24.421874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32163a4a-65f3-4325-9322-9eca3ad484e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:24.423487 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32163a4a-65f3-4325-9322-9eca3ad484e2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:24.437728 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 244/501] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:24.438710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32163a4a-65f3-4325-9322-9eca3ad484e2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:24.458572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245540e-d42e-4d6a-a667-535302a13d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:24.459210 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245540e-d42e-4d6a-a667-535302a13d79 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:24.459493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245540e-d42e-4d6a-a667-535302a13d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:24.459769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245540e-d42e-4d6a-a667-535302a13d79 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:24.460495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245540e-d42e-4d6a-a667-535302a13d79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:24.460807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245540e-d42e-4d6a-a667-535302a13d79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:24.462267 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245540e-d42e-4d6a-a667-535302a13d79 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:24.475975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 258/502] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:24.476863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245540e-d42e-4d6a-a667-535302a13d79 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:24.528660 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:24.589458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245540e-d42e-4d6a-a667-535302a13d79 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:24.605537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32163a4a-65f3-4325-9322-9eca3ad484e2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:24.624891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831a11d-ff02-4c00-b5b7-bce935e7662b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:24.625502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831a11d-ff02-4c00-b5b7-bce935e7662b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:24.625796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831a11d-ff02-4c00-b5b7-bce935e7662b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:24.626070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831a11d-ff02-4c00-b5b7-bce935e7662b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:24.643282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831a11d-ff02-4c00-b5b7-bce935e7662b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:24.646117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831a11d-ff02-4c00-b5b7-bce935e7662b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:24.650529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:24.664539 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b7ca2-1e2c-4825-865d-d42debb91215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:24.665550 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b7ca2-1e2c-4825-865d-d42debb91215 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:24.666021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b7ca2-1e2c-4825-865d-d42debb91215 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:24.666507 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b7ca2-1e2c-4825-865d-d42debb91215 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:24.667896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b7ca2-1e2c-4825-865d-d42debb91215 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:24.668417 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b7ca2-1e2c-4825-865d-d42debb91215 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:24.671033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b7ca2-1e2c-4825-865d-d42debb91215 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:24.673555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-5831a11d-ff02-4c00-b5b7-bce935e7662b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:24.673555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:24.673555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:24.673555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:24.673555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:24.673555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:24.673555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:24.673555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:24.673555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:24.673555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:24.673555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:24.673555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:24.673555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:24.673555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:24.673555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:24.675332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:24.677840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:24.677840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:24.677840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:24.677840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:24.677840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 259/503] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:24.677840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe89c5-383c-48bb-b852-45337df3c6c5 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:24.678466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe89c5-383c-48bb-b852-45337df3c6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:24.678466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe89c5-383c-48bb-b852-45337df3c6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:24.701644 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-86fe89c5-383c-48bb-b852-45337df3c6c5 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:24.702867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-86fe89c5-383c-48bb-b852-45337df3c6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:24.722961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 245/504] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Aug 3 10:30:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:24.724462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b7ca2-1e2c-4825-865d-d42debb91215 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:24.778823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 260/505] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:24.779862 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe89c5-383c-48bb-b852-45337df3c6c5 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:24.867811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b7ca2-1e2c-4825-865d-d42debb91215 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:24.883898 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:24.914902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd2f87-efb1-46fc-b202-88d4324d64bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:24.915862 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd2f87-efb1-46fc-b202-88d4324d64bf None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:24.916371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd2f87-efb1-46fc-b202-88d4324d64bf None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:24.916798 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd2f87-efb1-46fc-b202-88d4324d64bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:24.930497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd2f87-efb1-46fc-b202-88d4324d64bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:24.931941 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe89c5-383c-48bb-b852-45337df3c6c5 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:24.933033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd2f87-efb1-46fc-b202-88d4324d64bf None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-b2dd2f87-efb1-46fc-b202-88d4324d64bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:24.960556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:24.962402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:24.964500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:24.964500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:24.964500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:24.964500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:24.964500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:24.964500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:24.964500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:24.964500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:24.964500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:30:24.964500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:24.964500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 246/506] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 3 10:30:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:24.965678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd2f87-efb1-46fc-b202-88d4324d64bf None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:24.970470 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:24.986460 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8787ab-5e3c-4cad-9d50-28e0a66b87da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:24.987639 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8787ab-5e3c-4cad-9d50-28e0a66b87da None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:24.988171 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8787ab-5e3c-4cad-9d50-28e0a66b87da None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:24.988688 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8787ab-5e3c-4cad-9d50-28e0a66b87da None admin] PATH_INFO: `/v3/users/glance` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:24.997599 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8787ab-5e3c-4cad-9d50-28e0a66b87da None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:24.999095 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8787ab-5e3c-4cad-9d50-28e0a66b87da None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ca8787ab-5e3c-4cad-9d50-28e0a66b87da None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:25.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:25.016915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:25.017873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:25.017873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:25.017873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:25.017873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:25.017873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:25.017873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:25.017873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:25.017873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:25.017873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 03 10:30:25.017873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:25.020038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 261/507] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:25.020168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8787ab-5e3c-4cad-9d50-28e0a66b87da None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:25.114830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd2f87-efb1-46fc-b202-88d4324d64bf None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:25.140328 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8787ab-5e3c-4cad-9d50-28e0a66b87da None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:25.160118 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625f838-0226-499f-b182-b0492c0269bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:25.161290 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625f838-0226-499f-b182-b0492c0269bd None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:25.161708 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625f838-0226-499f-b182-b0492c0269bd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:25.162373 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625f838-0226-499f-b182-b0492c0269bd None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:25.163034 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625f838-0226-499f-b182-b0492c0269bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:25.163544 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625f838-0226-499f-b182-b0492c0269bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:25.167736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625f838-0226-499f-b182-b0492c0269bd None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:25.182998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 247/508] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:25.184485 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625f838-0226-499f-b182-b0492c0269bd None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:25.188462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fbf94a-65cd-42ea-8763-a5b75e6c6ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:25.189834 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbf94a-65cd-42ea-8763-a5b75e6c6ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:25.191047 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbf94a-65cd-42ea-8763-a5b75e6c6ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:25.191623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbf94a-65cd-42ea-8763-a5b75e6c6ac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:25.229592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:25.271807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:25.296702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fbf94a-65cd-42ea-8763-a5b75e6c6ac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:25.298519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fbf94a-65cd-42ea-8763-a5b75e6c6ac4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:25.328518 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625f838-0226-499f-b182-b0492c0269bd None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:25.377110 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b4098-7579-4f44-ad63-6b19778cd4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:25.378496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b4098-7579-4f44-ad63-6b19778cd4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:25.378967 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b4098-7579-4f44-ad63-6b19778cd4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:25.379534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b4098-7579-4f44-ad63-6b19778cd4f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:25.399676 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b4098-7579-4f44-ad63-6b19778cd4f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-2e8b4098-7579-4f44-ad63-6b19778cd4f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:25.408274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:25.410088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:25.412129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:25.412129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:25.412129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:25.412129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:25.412129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:25.412129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:25.412129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:25.412129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:25.412129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:25.412129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:25.412129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:25.412129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:25.412129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 248/509] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:25.412129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b4098-7579-4f44-ad63-6b19778cd4f3 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:25.446419 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 262/510] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:25.447780 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fbf94a-65cd-42ea-8763-a5b75e6c6ac4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:25.554783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:25.557786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:25.567985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b4098-7579-4f44-ad63-6b19778cd4f3 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:25.574980 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fbf94a-65cd-42ea-8763-a5b75e6c6ac4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:25.619351 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7300b7-e6e5-429d-a5ed-cf33c2061223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:25.620465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7300b7-e6e5-429d-a5ed-cf33c2061223 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:25.621005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7300b7-e6e5-429d-a5ed-cf33c2061223 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:25.621591 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7300b7-e6e5-429d-a5ed-cf33c2061223 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:25.623072 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7300b7-e6e5-429d-a5ed-cf33c2061223 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:25.623323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2d46e-dd84-4b62-9c47-b12f6fc4deed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:25.623747 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7300b7-e6e5-429d-a5ed-cf33c2061223 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:25.624305 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2d46e-dd84-4b62-9c47-b12f6fc4deed None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:25.624766 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2d46e-dd84-4b62-9c47-b12f6fc4deed None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:25.625251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2d46e-dd84-4b62-9c47-b12f6fc4deed None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:25.626621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7300b7-e6e5-429d-a5ed-cf33c2061223 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:25.626769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2d46e-dd84-4b62-9c47-b12f6fc4deed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:25.627208 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2d46e-dd84-4b62-9c47-b12f6fc4deed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:25.630323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2d46e-dd84-4b62-9c47-b12f6fc4deed None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:25.656312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 249/511] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 3 10:30:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:25.657737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2d46e-dd84-4b62-9c47-b12f6fc4deed None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:25.680288 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 263/512] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Aug 3 10:30:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:25.681819 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7300b7-e6e5-429d-a5ed-cf33c2061223 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:25.804819 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7300b7-e6e5-429d-a5ed-cf33c2061223 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:25.806027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2d46e-dd84-4b62-9c47-b12f6fc4deed None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:25.847272 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54680f94-3235-44b5-b3f9-9fbf5efc4f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:25.847896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54680f94-3235-44b5-b3f9-9fbf5efc4f79 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:25.848179 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54680f94-3235-44b5-b3f9-9fbf5efc4f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:25.848451 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54680f94-3235-44b5-b3f9-9fbf5efc4f79 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:25.849140 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54680f94-3235-44b5-b3f9-9fbf5efc4f79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:25.850651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54680f94-3235-44b5-b3f9-9fbf5efc4f79 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:25.852950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab468dd6-b0e5-4267-9b8e-479daf56dc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:25.854198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab468dd6-b0e5-4267-9b8e-479daf56dc7d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:25.854697 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab468dd6-b0e5-4267-9b8e-479daf56dc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:25.855183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab468dd6-b0e5-4267-9b8e-479daf56dc7d None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:25.856547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab468dd6-b0e5-4267-9b8e-479daf56dc7d None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:25.857188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab468dd6-b0e5-4267-9b8e-479daf56dc7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:25.860050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab468dd6-b0e5-4267-9b8e-479daf56dc7d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:25.877080 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 264/513] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:25.878737 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab468dd6-b0e5-4267-9b8e-479daf56dc7d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:25.889542 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 250/514] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:30:25 2022] POST /identity/v3/users => generated 247 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:30:25.889542 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54680f94-3235-44b5-b3f9-9fbf5efc4f79 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:25.902120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:25.906178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:26.069729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54680f94-3235-44b5-b3f9-9fbf5efc4f79 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:26.076321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab468dd6-b0e5-4267-9b8e-479daf56dc7d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:26.123483 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fe41a-db8f-40cd-ad5a-48ba974c850d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:26.124369 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fe41a-db8f-40cd-ad5a-48ba974c850d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:26.124935 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fe41a-db8f-40cd-ad5a-48ba974c850d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:26.125429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fe41a-db8f-40cd-ad5a-48ba974c850d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:26.147799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5597fa1-65c0-4f78-a878-b75f0d68695d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:26.148649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5597fa1-65c0-4f78-a878-b75f0d68695d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:26.149059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5597fa1-65c0-4f78-a878-b75f0d68695d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:26.149491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5597fa1-65c0-4f78-a878-b75f0d68695d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:26.163525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5597fa1-65c0-4f78-a878-b75f0d68695d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:26.165666 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5597fa1-65c0-4f78-a878-b75f0d68695d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:26.184054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-c5597fa1-65c0-4f78-a878-b75f0d68695d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:26.184054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:26.184054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:26.184054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:26.184054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:26.184054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:26.184054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:26.184054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:26.184054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:26.184054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:26.184054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:26.184054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:26.184054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:26.184054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:26.184054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:26.185706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:26.188037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:26.188037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:26.188037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:26.188037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:26.188037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 265/515] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:26.188923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5597fa1-65c0-4f78-a878-b75f0d68695d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:26.259143 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fe41a-db8f-40cd-ad5a-48ba974c850d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:26.262014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fe41a-db8f-40cd-ad5a-48ba974c850d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:26.274052 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:26.282642 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:26.357999 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5597fa1-65c0-4f78-a878-b75f0d68695d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:26.415029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef68a5-859b-4d89-9501-67c0af515525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:26.415697 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef68a5-859b-4d89-9501-67c0af515525 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:26.415976 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef68a5-859b-4d89-9501-67c0af515525 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:26.416249 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef68a5-859b-4d89-9501-67c0af515525 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:26.450335 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 251/516] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:26.451827 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fe41a-db8f-40cd-ad5a-48ba974c850d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:26.526657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef68a5-859b-4d89-9501-67c0af515525 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:26.528254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef68a5-859b-4d89-9501-67c0af515525 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:26.636693 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:26.654554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fe41a-db8f-40cd-ad5a-48ba974c850d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:26.673776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:26.724533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 266/517] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:26.724533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef68a5-859b-4d89-9501-67c0af515525 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:26.726220 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911ea8b-2410-4ff7-8e56-ae1a38bb7062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:26.727085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911ea8b-2410-4ff7-8e56-ae1a38bb7062 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:26.727619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911ea8b-2410-4ff7-8e56-ae1a38bb7062 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:26.728701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911ea8b-2410-4ff7-8e56-ae1a38bb7062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:26.742261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911ea8b-2410-4ff7-8e56-ae1a38bb7062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:26.744199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911ea8b-2410-4ff7-8e56-ae1a38bb7062 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:26.767678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-b911ea8b-2410-4ff7-8e56-ae1a38bb7062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:26.767678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:26.767678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:26.767678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:26.767678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:26.767678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:26.767678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:26.767678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:26.767678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:26.767678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:26.767678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:26.767678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:26.767678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:26.767678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:26.767678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:26.769757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:26.771513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:26.771513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:26.771513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:26.771513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:26.771513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 252/518] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:26.772054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911ea8b-2410-4ff7-8e56-ae1a38bb7062 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:26.888905 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef68a5-859b-4d89-9501-67c0af515525 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:26.938288 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa6cd7-d7be-4d9b-b157-003070d81d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:26.939343 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa6cd7-d7be-4d9b-b157-003070d81d7a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:26.939750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa6cd7-d7be-4d9b-b157-003070d81d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:26.940224 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa6cd7-d7be-4d9b-b157-003070d81d7a None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:26.941291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa6cd7-d7be-4d9b-b157-003070d81d7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:26.941754 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa6cd7-d7be-4d9b-b157-003070d81d7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:26.943898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa6cd7-d7be-4d9b-b157-003070d81d7a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:26.958962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911ea8b-2410-4ff7-8e56-ae1a38bb7062 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:26.965705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 267/519] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:26.967402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa6cd7-d7be-4d9b-b157-003070d81d7a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:26.992582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:27.027840 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7041c40-e13f-495a-b220-b12e699ba229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:27.028980 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7041c40-e13f-495a-b220-b12e699ba229 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:27.029636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7041c40-e13f-495a-b220-b12e699ba229 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:27.030151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7041c40-e13f-495a-b220-b12e699ba229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:27.051503 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7041c40-e13f-495a-b220-b12e699ba229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:27.054434 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7041c40-e13f-495a-b220-b12e699ba229 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:27.083899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-b7041c40-e13f-495a-b220-b12e699ba229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:27.083899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:27.083899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:27.083899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:27.083899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:27.083899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:27.083899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:27.083899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:27.083899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:27.083899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:27.083899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:27.083899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:27.083899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:27.083899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:27.083899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:27.084824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:27.085909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:27.085909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:27.085909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:27.085909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:27.089739 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 253/520] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:27.091497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7041c40-e13f-495a-b220-b12e699ba229 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:27.117197 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa6cd7-d7be-4d9b-b157-003070d81d7a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:27.120345 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:27.177804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35811f9-08c7-4538-b750-dcde068bbcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:27.178659 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35811f9-08c7-4538-b750-dcde068bbcaa None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:27.179062 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35811f9-08c7-4538-b750-dcde068bbcaa None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:27.179470 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35811f9-08c7-4538-b750-dcde068bbcaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:27.194741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35811f9-08c7-4538-b750-dcde068bbcaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:27.196487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35811f9-08c7-4538-b750-dcde068bbcaa None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:27.215583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-f35811f9-08c7-4538-b750-dcde068bbcaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:27.215583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:27.215583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:27.215583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:27.215583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:27.215583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:27.215583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:27.215583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:27.215583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:27.215583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:27.215583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:27.215583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:27.215583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:27.215583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:27.215583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:27.217253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:27.218952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:27.218952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:27.218952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:27.218952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:27.218952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 268/521] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:27.220559 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35811f9-08c7-4538-b750-dcde068bbcaa None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:27.275762 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7041c40-e13f-495a-b220-b12e699ba229 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:27.324310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433068c-3c54-40f9-a749-35aebbf47fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:27.325082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433068c-3c54-40f9-a749-35aebbf47fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:27.325507 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433068c-3c54-40f9-a749-35aebbf47fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:27.325933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433068c-3c54-40f9-a749-35aebbf47fb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:27.326864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433068c-3c54-40f9-a749-35aebbf47fb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:27.327351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433068c-3c54-40f9-a749-35aebbf47fb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:27.328908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433068c-3c54-40f9-a749-35aebbf47fb8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:27.355725 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:27.356889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 254/522] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:27.358801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433068c-3c54-40f9-a749-35aebbf47fb8 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:27.404248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35811f9-08c7-4538-b750-dcde068bbcaa None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:27.469165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba6469-036f-412f-9683-5c6c162a07f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:27.470332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba6469-036f-412f-9683-5c6c162a07f7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:27.470756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba6469-036f-412f-9683-5c6c162a07f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:27.471154 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba6469-036f-412f-9683-5c6c162a07f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:27.483747 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ba6469-036f-412f-9683-5c6c162a07f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:27.484097 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:27.486291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ba6469-036f-412f-9683-5c6c162a07f7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:27.508864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-b1ba6469-036f-412f-9683-5c6c162a07f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:27.508864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:27.508864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:27.508864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:27.508864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:27.508864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:27.508864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:27.508864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:27.508864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:27.508864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:27.508864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:27.508864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:27.508864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:27.508864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:27.508864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:27.510818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:27.512525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:27.512525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:27.512525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:27.512525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:27.512525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:27.512525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:27.512525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:27.512525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:27.512525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 269/523] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:27.512525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba6469-036f-412f-9683-5c6c162a07f7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:27.554757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433068c-3c54-40f9-a749-35aebbf47fb8 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:27.617546 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959d34b-1427-4372-accb-89836d2e8e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:27.618478 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959d34b-1427-4372-accb-89836d2e8e90 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:27.619205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959d34b-1427-4372-accb-89836d2e8e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:27.619486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959d34b-1427-4372-accb-89836d2e8e90 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:27.620256 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959d34b-1427-4372-accb-89836d2e8e90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:27.620565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959d34b-1427-4372-accb-89836d2e8e90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:27.622025 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959d34b-1427-4372-accb-89836d2e8e90 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:27.638760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 255/524] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:27.638760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959d34b-1427-4372-accb-89836d2e8e90 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:27.697040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba6469-036f-412f-9683-5c6c162a07f7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:27.750489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:27.766232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcf70c-50ae-4723-b7e9-c327f3acd672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:27.767230 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcf70c-50ae-4723-b7e9-c327f3acd672 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:27.767706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcf70c-50ae-4723-b7e9-c327f3acd672 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:27.768678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcf70c-50ae-4723-b7e9-c327f3acd672 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:27.769958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dcf70c-50ae-4723-b7e9-c327f3acd672 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:27.770552 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dcf70c-50ae-4723-b7e9-c327f3acd672 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:27.772944 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dcf70c-50ae-4723-b7e9-c327f3acd672 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:27.805678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 270/525] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:27.807486 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcf70c-50ae-4723-b7e9-c327f3acd672 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:27.844666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959d34b-1427-4372-accb-89836d2e8e90 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:27.886770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:27.894863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a68fd8-4abb-4bd0-acae-790ba87bb665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:27.897326 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a68fd8-4abb-4bd0-acae-790ba87bb665 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:27.900207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a68fd8-4abb-4bd0-acae-790ba87bb665 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:27.900628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a68fd8-4abb-4bd0-acae-790ba87bb665 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:27.913723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a68fd8-4abb-4bd0-acae-790ba87bb665 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:27.915310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a68fd8-4abb-4bd0-acae-790ba87bb665 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-75a68fd8-4abb-4bd0-acae-790ba87bb665 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:27.928984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:27.930008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:27.930894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:27.930894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:27.930894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:27.930894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:27.930894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:27.930894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:27.930894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:27.930894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:27.930894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 10:30:27.930894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:27.930894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 256/526] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:27.931535 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a68fd8-4abb-4bd0-acae-790ba87bb665 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:27.992598 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcf70c-50ae-4723-b7e9-c327f3acd672 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:28.055548 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6e6d3-eafd-4fa4-90c0-b2be72860b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:28.056705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6e6d3-eafd-4fa4-90c0-b2be72860b0a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:28.057207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6e6d3-eafd-4fa4-90c0-b2be72860b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:28.057956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6e6d3-eafd-4fa4-90c0-b2be72860b0a None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:28.059014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6e6d3-eafd-4fa4-90c0-b2be72860b0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:28.059534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6e6d3-eafd-4fa4-90c0-b2be72860b0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:28.061760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6e6d3-eafd-4fa4-90c0-b2be72860b0a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:28.093762 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a68fd8-4abb-4bd0-acae-790ba87bb665 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:28.096766 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 271/527] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:28.099265 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6e6d3-eafd-4fa4-90c0-b2be72860b0a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:28.160914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd858f5f-85b6-4d0e-bd96-9dc2aa71af58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:28.161639 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd858f5f-85b6-4d0e-bd96-9dc2aa71af58 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:28.161925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd858f5f-85b6-4d0e-bd96-9dc2aa71af58 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:28.162535 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd858f5f-85b6-4d0e-bd96-9dc2aa71af58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:28.169974 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:28.170726 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd858f5f-85b6-4d0e-bd96-9dc2aa71af58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:28.172207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd858f5f-85b6-4d0e-bd96-9dc2aa71af58 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:28.187631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-dd858f5f-85b6-4d0e-bd96-9dc2aa71af58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:28.187631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:28.187631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:28.187631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:28.187631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:28.187631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:28.187631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:28.187631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:28.187631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:28.187631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:28.187631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:28.187631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:28.187631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:28.187631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:28.187631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:28.189066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:28.190177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:28.190177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:28.190177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:28.190177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:28.190177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:28.190177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:28.190177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:28.190177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:28.190177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 257/528] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:28.190804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd858f5f-85b6-4d0e-bd96-9dc2aa71af58 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:28.245063 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6e6d3-eafd-4fa4-90c0-b2be72860b0a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:28.251708 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:28.288887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0014596-5435-4d5b-9bc8-2173a0c0c8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:28.289920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0014596-5435-4d5b-9bc8-2173a0c0c8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:28.290414 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0014596-5435-4d5b-9bc8-2173a0c0c8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:28.290926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0014596-5435-4d5b-9bc8-2173a0c0c8f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:28.303379 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0014596-5435-4d5b-9bc8-2173a0c0c8f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:28.304294 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd858f5f-85b6-4d0e-bd96-9dc2aa71af58 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:28.306221 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0014596-5435-4d5b-9bc8-2173a0c0c8f1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:28.323051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-b0014596-5435-4d5b-9bc8-2173a0c0c8f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:28.323051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:28.323051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:28.323051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:28.323051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:28.323051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:28.323051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:28.323051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:28.323051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:28.323051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:28.323051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:28.323051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:28.323051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:28.323051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:28.323051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:28.323970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:28.324843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:28.324843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:28.324843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:28.324843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:28.324843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:28.324843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:28.324843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:28.324843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:28.324843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 272/529] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:28.325306 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0014596-5435-4d5b-9bc8-2173a0c0c8f1 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:28.345842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf6a96-bc56-466f-9838-c2019091325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:28.346981 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf6a96-bc56-466f-9838-c2019091325a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:28.347253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf6a96-bc56-466f-9838-c2019091325a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:28.347524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf6a96-bc56-466f-9838-c2019091325a None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:28.348330 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf6a96-bc56-466f-9838-c2019091325a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:28.348623 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf6a96-bc56-466f-9838-c2019091325a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:28.350114 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf6a96-bc56-466f-9838-c2019091325a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:28.370731 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 258/530] 10.222.0.47 () {56 vars in 1139 bytes} [Wed Aug 3 10:30:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:28.372009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29eeef-9144-44df-8732-468da1418d01 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:28.372295 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29eeef-9144-44df-8732-468da1418d01 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:28.372572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29eeef-9144-44df-8732-468da1418d01 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:28.376095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 259/531] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:28.376095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29eeef-9144-44df-8732-468da1418d01 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:28.457572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0014596-5435-4d5b-9bc8-2173a0c0c8f1 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:28.469845 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:28.502314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dad68a-696b-4d03-a234-2acefd6d1815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:28.502972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dad68a-696b-4d03-a234-2acefd6d1815 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:28.503259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dad68a-696b-4d03-a234-2acefd6d1815 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:28.503534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dad68a-696b-4d03-a234-2acefd6d1815 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:28.504339 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dad68a-696b-4d03-a234-2acefd6d1815 None admin] RBAC: Adding query filter params (role.id=1df33ad923a44a59b50dcacb6a75fefb, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=3b5f547a388c4f64aeb95a7435818741) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:28.504645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dad68a-696b-4d03-a234-2acefd6d1815 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:28.506128 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dad68a-696b-4d03-a234-2acefd6d1815 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:28.507942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29eeef-9144-44df-8732-468da1418d01 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:28.512697 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 273/532] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:30:28 2022] GET /identity/v3/role_assignments?user.id=3b5f547a388c4f64aeb95a7435818741&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=1df33ad923a44a59b50dcacb6a75fefb => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:28.513586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dad68a-696b-4d03-a234-2acefd6d1815 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:28.528376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:28.567184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd7e99-8103-4265-a1e9-c2fd83d36f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:28.567711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd7e99-8103-4265-a1e9-c2fd83d36f5d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:28.568245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd7e99-8103-4265-a1e9-c2fd83d36f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:28.568514 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd7e99-8103-4265-a1e9-c2fd83d36f5d None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:28.569251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd7e99-8103-4265-a1e9-c2fd83d36f5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:28.569599 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd7e99-8103-4265-a1e9-c2fd83d36f5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:28.571150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd7e99-8103-4265-a1e9-c2fd83d36f5d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:28.592745 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 260/533] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:28.593430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd7e99-8103-4265-a1e9-c2fd83d36f5d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:28.690485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dad68a-696b-4d03-a234-2acefd6d1815 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:28.732928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3f3d6-89bc-47bf-ac34-1421567c5cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:28.733740 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3f3d6-89bc-47bf-ac34-1421567c5cea None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:28.733998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3f3d6-89bc-47bf-ac34-1421567c5cea None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:28.734153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3f3d6-89bc-47bf-ac34-1421567c5cea None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:28.734958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3f3d6-89bc-47bf-ac34-1421567c5cea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:28.735274 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3f3d6-89bc-47bf-ac34-1421567c5cea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:28.736785 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3f3d6-89bc-47bf-ac34-1421567c5cea None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:28.751866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 274/534] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:28.753266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34340d7e-3085-4ab0-a922-a2943d1ddeb9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:28.753582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34340d7e-3085-4ab0-a922-a2943d1ddeb9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:28.753864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34340d7e-3085-4ab0-a922-a2943d1ddeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:28.754704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd7e99-8103-4265-a1e9-c2fd83d36f5d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:28.777923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-34340d7e-3085-4ab0-a922-a2943d1ddeb9 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:28.781186 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-34340d7e-3085-4ab0-a922-a2943d1ddeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:28.807522 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9df2e-e560-4b93-a320-6ba9fd774735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:28.808979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9df2e-e560-4b93-a320-6ba9fd774735 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:28.809277 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9df2e-e560-4b93-a320-6ba9fd774735 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:28.809615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9df2e-e560-4b93-a320-6ba9fd774735 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:28.822232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:28.867889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:28.901507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 275/535] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:28.901507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34340d7e-3085-4ab0-a922-a2943d1ddeb9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:28.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9df2e-e560-4b93-a320-6ba9fd774735 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:28.909755 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9df2e-e560-4b93-a320-6ba9fd774735 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:29.058530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34340d7e-3085-4ab0-a922-a2943d1ddeb9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:29.086852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 261/536] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:29.088306 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9df2e-e560-4b93-a320-6ba9fd774735 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:29.117027 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66cbf1-b705-4405-a23a-de3c1b25c179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:29.118157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66cbf1-b705-4405-a23a-de3c1b25c179 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:29.118785 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66cbf1-b705-4405-a23a-de3c1b25c179 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:29.119161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66cbf1-b705-4405-a23a-de3c1b25c179 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:29.120086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66cbf1-b705-4405-a23a-de3c1b25c179 None admin] RBAC: Adding query filter params (role.id=1df33ad923a44a59b50dcacb6a75fefb, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=e66f25cc329f418eba6bd637bf3e8a3a) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:29.120507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66cbf1-b705-4405-a23a-de3c1b25c179 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:29.122116 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66cbf1-b705-4405-a23a-de3c1b25c179 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:29.141959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 276/537] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:30:28 2022] GET /identity/v3/role_assignments?user.id=e66f25cc329f418eba6bd637bf3e8a3a&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=1df33ad923a44a59b50dcacb6a75fefb => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:29.142947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66cbf1-b705-4405-a23a-de3c1b25c179 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:29.211073 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:29.250964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:29.324739 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9df2e-e560-4b93-a320-6ba9fd774735 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:29.353857 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66cbf1-b705-4405-a23a-de3c1b25c179 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:29.397369 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628dfe2e-625e-4bee-85cd-f6d5084733aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:29.398596 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba539ea-5b4a-4679-85c0-fa037979894b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:29.399025 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628dfe2e-625e-4bee-85cd-f6d5084733aa None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:29.399455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba539ea-5b4a-4679-85c0-fa037979894b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:29.399863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba539ea-5b4a-4679-85c0-fa037979894b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:29.400185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628dfe2e-625e-4bee-85cd-f6d5084733aa None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:29.400347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba539ea-5b4a-4679-85c0-fa037979894b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:29.401038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628dfe2e-625e-4bee-85cd-f6d5084733aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:29.402976 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628dfe2e-625e-4bee-85cd-f6d5084733aa None admin] RBAC: Adding query filter params (role.id=1df33ad923a44a59b50dcacb6a75fefb, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=300fb5e1c5c546808b935871673e3d0f) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:29.403791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628dfe2e-625e-4bee-85cd-f6d5084733aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:29.407065 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628dfe2e-625e-4bee-85cd-f6d5084733aa None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:29.416934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba539ea-5b4a-4679-85c0-fa037979894b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-2ba539ea-5b4a-4679-85c0-fa037979894b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:29.425503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:29.427555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:29.429138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:29.429138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:29.429138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:29.429138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:29.429138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:29.429138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:29.429138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:29.429138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:29.429138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:29.429138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:29.429138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:29.429138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:29.430508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 277/539] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:29.430721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 262/539] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:30:29 2022] GET /identity/v3/role_assignments?user.id=300fb5e1c5c546808b935871673e3d0f&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=1df33ad923a44a59b50dcacb6a75fefb => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:29.436302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba539ea-5b4a-4679-85c0-fa037979894b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:29.439179 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628dfe2e-625e-4bee-85cd-f6d5084733aa None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:29.577516 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba539ea-5b4a-4679-85c0-fa037979894b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:29.594704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:29.618967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6464da-daae-4705-96ff-4dd366d7074e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:29.620530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6464da-daae-4705-96ff-4dd366d7074e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:29.621062 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6464da-daae-4705-96ff-4dd366d7074e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:29.621450 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6464da-daae-4705-96ff-4dd366d7074e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:29.655773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6464da-daae-4705-96ff-4dd366d7074e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:29.655773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6464da-daae-4705-96ff-4dd366d7074e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:29.707009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-fe6464da-daae-4705-96ff-4dd366d7074e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:29.707009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:29.707009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:29.707009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:29.707009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:29.707009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:29.707009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:29.707009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:29.707009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:29.707009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:29.707009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:29.707009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:29.707009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:29.707009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:29.707009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:29.708514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:29.710042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:29.710042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:29.710042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:29.710042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:29.714402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 278/540] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:29.736640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6464da-daae-4705-96ff-4dd366d7074e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:29.807266 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628dfe2e-625e-4bee-85cd-f6d5084733aa None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:29.873764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cbf9c5-77c6-4d9c-bb22-9fbc9afe90ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:29.874752 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbf9c5-77c6-4d9c-bb22-9fbc9afe90ea None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:29.875246 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbf9c5-77c6-4d9c-bb22-9fbc9afe90ea None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:29.875752 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbf9c5-77c6-4d9c-bb22-9fbc9afe90ea None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:29.877091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cbf9c5-77c6-4d9c-bb22-9fbc9afe90ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:29.877698 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cbf9c5-77c6-4d9c-bb22-9fbc9afe90ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:29.885723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cbf9c5-77c6-4d9c-bb22-9fbc9afe90ea None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:29.895707 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:29.902619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 263/541] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:29.917888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cbf9c5-77c6-4d9c-bb22-9fbc9afe90ea None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:30.095459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6464da-daae-4705-96ff-4dd366d7074e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:30.146201 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cbf9c5-77c6-4d9c-bb22-9fbc9afe90ea None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:30.182399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe802b-0f1c-4733-867b-07175f971ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:30.183453 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe802b-0f1c-4733-867b-07175f971ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:30.183937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe802b-0f1c-4733-867b-07175f971ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:30.184408 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe802b-0f1c-4733-867b-07175f971ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:30.185346 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe802b-0f1c-4733-867b-07175f971ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:30.185880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe802b-0f1c-4733-867b-07175f971ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:30.187797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe802b-0f1c-4733-867b-07175f971ad8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:30.200977 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:30.207541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 279/542] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 472 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:30:30.218892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe802b-0f1c-4733-867b-07175f971ad8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:30.226612 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d500d5b-3908-47b5-b2a3-cbae93f38f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:30.227684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d500d5b-3908-47b5-b2a3-cbae93f38f92 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:30.228176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d500d5b-3908-47b5-b2a3-cbae93f38f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:30.228640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d500d5b-3908-47b5-b2a3-cbae93f38f92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:30.253508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d500d5b-3908-47b5-b2a3-cbae93f38f92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:30.253508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d500d5b-3908-47b5-b2a3-cbae93f38f92 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:30.287615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-0d500d5b-3908-47b5-b2a3-cbae93f38f92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:30.287615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:30.287615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:30.287615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:30.287615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:30.287615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:30.287615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:30.287615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:30.287615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:30.287615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:30.287615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:30.287615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:30.287615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:30.287615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:30.287615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:30.288527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:30.289394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:30.289394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:30.289394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:30.289394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:30.290194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 264/543] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:30.299014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d500d5b-3908-47b5-b2a3-cbae93f38f92 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:30.403125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:30.510968 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d500d5b-3908-47b5-b2a3-cbae93f38f92 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:30.567481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe802b-0f1c-4733-867b-07175f971ad8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:30.575961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318cb6e3-991c-48ae-b6b2-26287a61060d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:30.577012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318cb6e3-991c-48ae-b6b2-26287a61060d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:30.577561 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318cb6e3-991c-48ae-b6b2-26287a61060d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:30.578055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318cb6e3-991c-48ae-b6b2-26287a61060d None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:30.579317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318cb6e3-991c-48ae-b6b2-26287a61060d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:30.579867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318cb6e3-991c-48ae-b6b2-26287a61060d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:30.583167 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318cb6e3-991c-48ae-b6b2-26287a61060d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:30.620141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 265/544] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:30.634704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318cb6e3-991c-48ae-b6b2-26287a61060d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:30.640943 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537b4d4-691d-4552-87cb-b34181c94ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:30.641946 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537b4d4-691d-4552-87cb-b34181c94ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:30.642455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537b4d4-691d-4552-87cb-b34181c94ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:30.642923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537b4d4-691d-4552-87cb-b34181c94ae8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:30.652722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537b4d4-691d-4552-87cb-b34181c94ae8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:30.654784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537b4d4-691d-4552-87cb-b34181c94ae8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:30.667669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-2537b4d4-691d-4552-87cb-b34181c94ae8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:30.667669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:30.667669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:30.667669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:30.667669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:30.667669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:30.667669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:30.667669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:30.667669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:30.667669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:30.667669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:30.667669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:30.667669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:30.667669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:30.667669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:30.668574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:30.669427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:30.669427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:30.669427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:30.669427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:30.669427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:30.669427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:30.669427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:30.669427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:30.671990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 280/545] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 454 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:30.686278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537b4d4-691d-4552-87cb-b34181c94ae8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:30.791165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318cb6e3-991c-48ae-b6b2-26287a61060d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:30.808276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:30.831075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9e865-da66-4922-bebf-00f3980875cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:30.832106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9e865-da66-4922-bebf-00f3980875cf None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:30.832585 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9e865-da66-4922-bebf-00f3980875cf None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:30.833342 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9e865-da66-4922-bebf-00f3980875cf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:30.842646 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9e865-da66-4922-bebf-00f3980875cf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:30.844463 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9e865-da66-4922-bebf-00f3980875cf None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-5fd9e865-da66-4922-bebf-00f3980875cf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:30.860080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:30.861102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:30.863404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:30.863404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:30.863404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:30.863404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:30.863404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:30.863404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:30.863404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:30.863404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:30.863404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 10:30:30.863404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:30.866834 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 266/546] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:30:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:30.879157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9e865-da66-4922-bebf-00f3980875cf None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:30.910626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:31.033251 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537b4d4-691d-4552-87cb-b34181c94ae8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:31.035530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9e865-da66-4922-bebf-00f3980875cf None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:31.088256 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ae184-0993-4069-8f01-5ce7dbe6b574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:31.089646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ae184-0993-4069-8f01-5ce7dbe6b574 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:31.090177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ae184-0993-4069-8f01-5ce7dbe6b574 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:31.090650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ae184-0993-4069-8f01-5ce7dbe6b574 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:31.091704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ae184-0993-4069-8f01-5ce7dbe6b574 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:31.092298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ae184-0993-4069-8f01-5ce7dbe6b574 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:31.094202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ae184-0993-4069-8f01-5ce7dbe6b574 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:31.097560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c95e-1bff-43fd-845f-aa9de59e194d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:31.098199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c95e-1bff-43fd-845f-aa9de59e194d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:31.098485 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c95e-1bff-43fd-845f-aa9de59e194d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:31.098763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c95e-1bff-43fd-845f-aa9de59e194d None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:31.099569 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0c95e-1bff-43fd-845f-aa9de59e194d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:31.099877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0c95e-1bff-43fd-845f-aa9de59e194d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:31.101322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0c95e-1bff-43fd-845f-aa9de59e194d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:31.111223 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 281/547] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:31.113726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bef42-1526-4b23-9c02-c3ec61f248b0 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:31.114253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bef42-1526-4b23-9c02-c3ec61f248b0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:31.114724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bef42-1526-4b23-9c02-c3ec61f248b0 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:31.116457 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 282/548] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:31.121022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bef42-1526-4b23-9c02-c3ec61f248b0 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:31.127677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 267/549] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Aug 3 10:30:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:31.129326 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6166e9-c988-4484-9ca6-95ef43ecd521 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:31.129914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6166e9-c988-4484-9ca6-95ef43ecd521 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:31.130405 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6166e9-c988-4484-9ca6-95ef43ecd521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:31.153963 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-fd6166e9-c988-4484-9ca6-95ef43ecd521 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:31.155442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fd6166e9-c988-4484-9ca6-95ef43ecd521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:31.178626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:31.240680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bef42-1526-4b23-9c02-c3ec61f248b0 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:31.261636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 268/550] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:31.264115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6166e9-c988-4484-9ca6-95ef43ecd521 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:31.270683 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:31.301571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb250b0-df68-4c7c-96c5-554cbdd57504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:31.302207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb250b0-df68-4c7c-96c5-554cbdd57504 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:31.303139 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb250b0-df68-4c7c-96c5-554cbdd57504 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:31.303428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb250b0-df68-4c7c-96c5-554cbdd57504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:31.304429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb250b0-df68-4c7c-96c5-554cbdd57504 None admin] RBAC: Adding query filter params (role.id=1df33ad923a44a59b50dcacb6a75fefb, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=1c211cc19a2045bd9383d31c7e81ceac) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:31.304884 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb250b0-df68-4c7c-96c5-554cbdd57504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:31.306763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb250b0-df68-4c7c-96c5-554cbdd57504 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:31.315659 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 283/551] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:30:31 2022] GET /identity/v3/role_assignments?user.id=1c211cc19a2045bd9383d31c7e81ceac&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=1df33ad923a44a59b50dcacb6a75fefb => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:31.316784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb250b0-df68-4c7c-96c5-554cbdd57504 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:31.472723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6166e9-c988-4484-9ca6-95ef43ecd521 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:31.524246 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f504d-1077-48d0-ad3c-b818cfb4d80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:31.524982 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f504d-1077-48d0-ad3c-b818cfb4d80e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:31.525270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f504d-1077-48d0-ad3c-b818cfb4d80e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:31.525585 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f504d-1077-48d0-ad3c-b818cfb4d80e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:31.550561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb250b0-df68-4c7c-96c5-554cbdd57504 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:31.561511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:31.622235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f504d-1077-48d0-ad3c-b818cfb4d80e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:31.624151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f504d-1077-48d0-ad3c-b818cfb4d80e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:31.658326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8bb2e-340e-47c5-80d7-5e32034fffb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:31.659707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8bb2e-340e-47c5-80d7-5e32034fffb2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:31.660017 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8bb2e-340e-47c5-80d7-5e32034fffb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:31.660374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8bb2e-340e-47c5-80d7-5e32034fffb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:31.718210 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8bb2e-340e-47c5-80d7-5e32034fffb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-f3e8bb2e-340e-47c5-80d7-5e32034fffb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:31.724649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:31.725626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:31.726427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:31.726427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:31.726427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:31.726427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:31.726427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:31.726427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:31.726427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:31.726427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:31.726427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:31.726427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:31.726427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:31.726427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:31.733340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 284/552] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:31.734664 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8bb2e-340e-47c5-80d7-5e32034fffb2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:31.756866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 269/553] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:31.770873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f504d-1077-48d0-ad3c-b818cfb4d80e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:31.834948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:31.894113 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f504d-1077-48d0-ad3c-b818cfb4d80e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:31.910218 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:31.934875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812824f-fade-4597-bfea-7dc9df6e4dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:31.935836 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812824f-fade-4597-bfea-7dc9df6e4dee None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:31.936250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812824f-fade-4597-bfea-7dc9df6e4dee None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:31.936667 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812824f-fade-4597-bfea-7dc9df6e4dee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:31.948972 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812824f-fade-4597-bfea-7dc9df6e4dee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:31.950852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812824f-fade-4597-bfea-7dc9df6e4dee None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:31.973364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-a812824f-fade-4597-bfea-7dc9df6e4dee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:31.973364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:31.973364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:31.973364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:31.973364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:31.973364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:31.973364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:31.973364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:31.973364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:31.973364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:31.973364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:31.973364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:31.973364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:31.973364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:31.973364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:31.974327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:31.975226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:31.975226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:31.975226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:31.975226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:31.977457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 270/554] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:30:31.978999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463dc79-2660-418c-9e92-da30c754b684 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:31.979487 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463dc79-2660-418c-9e92-da30c754b684 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:31.979955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463dc79-2660-418c-9e92-da30c754b684 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:31.981791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 271/555] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:31.982971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463dc79-2660-418c-9e92-da30c754b684 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:32.015712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8bb2e-340e-47c5-80d7-5e32034fffb2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:32.091252 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c812c-ac35-4b51-83d8-7a8d87f5456a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:32.092267 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c812c-ac35-4b51-83d8-7a8d87f5456a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:32.092724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c812c-ac35-4b51-83d8-7a8d87f5456a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:32.093335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c812c-ac35-4b51-83d8-7a8d87f5456a None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:32.094912 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c812c-ac35-4b51-83d8-7a8d87f5456a None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:32.095359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c812c-ac35-4b51-83d8-7a8d87f5456a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:32.097957 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c812c-ac35-4b51-83d8-7a8d87f5456a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:32.105666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463dc79-2660-418c-9e92-da30c754b684 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:32.114237 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 285/556] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:30:32.116597 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa39e72-6006-438f-9ef9-52cf5550d466 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:32.116747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:32.116992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa39e72-6006-438f-9ef9-52cf5550d466 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:32.117504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa39e72-6006-438f-9ef9-52cf5550d466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:32.146472 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289bfb1-c30f-4588-b464-9b131cde896a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:32.147600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289bfb1-c30f-4588-b464-9b131cde896a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:32.148187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289bfb1-c30f-4588-b464-9b131cde896a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:32.148663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289bfb1-c30f-4588-b464-9b131cde896a None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:32.148958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3aa39e72-6006-438f-9ef9-52cf5550d466 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:32.149871 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289bfb1-c30f-4588-b464-9b131cde896a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:32.150381 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289bfb1-c30f-4588-b464-9b131cde896a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:32.150997 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3aa39e72-6006-438f-9ef9-52cf5550d466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:32.152235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289bfb1-c30f-4588-b464-9b131cde896a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:32.167243 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 272/557] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:32.168557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289bfb1-c30f-4588-b464-9b131cde896a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:32.283765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 286/558] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:32.285166 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa39e72-6006-438f-9ef9-52cf5550d466 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:32.287194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289bfb1-c30f-4588-b464-9b131cde896a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:32.330012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964ba0b-b58d-40f2-b3da-d72dd7b6b50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:32.331288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964ba0b-b58d-40f2-b3da-d72dd7b6b50e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:32.331643 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964ba0b-b58d-40f2-b3da-d72dd7b6b50e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:32.331927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964ba0b-b58d-40f2-b3da-d72dd7b6b50e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:32.343279 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964ba0b-b58d-40f2-b3da-d72dd7b6b50e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:32.344749 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964ba0b-b58d-40f2-b3da-d72dd7b6b50e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:32.356046 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:32.360363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-2964ba0b-b58d-40f2-b3da-d72dd7b6b50e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:32.360363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:32.360363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:32.360363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:32.360363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:32.360363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:32.360363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:32.360363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:32.360363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:32.360363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:32.360363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:32.360363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:32.360363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:32.360363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:32.360363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:32.362545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:32.363428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:32.363428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:32.363428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:32.363428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:32.367785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 273/559] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:32.369932 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8494c5-eca0-451c-acda-901defa3e6cc None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:32.370435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8494c5-eca0-451c-acda-901defa3e6cc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:32.371040 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8494c5-eca0-451c-acda-901defa3e6cc None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:32.373921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 274/560] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:32.375311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8494c5-eca0-451c-acda-901defa3e6cc None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:32.418171 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:32.462832 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa39e72-6006-438f-9ef9-52cf5550d466 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:32.523855 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d330e5-6ad0-435a-9dbd-f566c961f199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:32.525043 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d330e5-6ad0-435a-9dbd-f566c961f199 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:32.525687 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d330e5-6ad0-435a-9dbd-f566c961f199 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:32.526226 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d330e5-6ad0-435a-9dbd-f566c961f199 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:32.538763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d330e5-6ad0-435a-9dbd-f566c961f199 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:32.539298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8494c5-eca0-451c-acda-901defa3e6cc None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:32.541285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d330e5-6ad0-435a-9dbd-f566c961f199 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:32.563611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-e6d330e5-6ad0-435a-9dbd-f566c961f199 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:32.563611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:32.563611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:32.563611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:32.563611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:32.563611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:32.563611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:32.563611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:32.563611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:32.563611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:32.563611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:32.563611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:32.563611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:32.563611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:32.563611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:32.564607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:32.565704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:32.565704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:32.565704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:32.565704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:32.565704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:32.565704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:32.565704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:32.565704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:32.569827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 287/561] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:32.571624 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d330e5-6ad0-435a-9dbd-f566c961f199 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:32.592081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63930771-2578-4bb4-9952-fc39c0f82f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:32.593138 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63930771-2578-4bb4-9952-fc39c0f82f3d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:32.593431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63930771-2578-4bb4-9952-fc39c0f82f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:32.593747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63930771-2578-4bb4-9952-fc39c0f82f3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:32.614936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63930771-2578-4bb4-9952-fc39c0f82f3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-63930771-2578-4bb4-9952-fc39c0f82f3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:32.624783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:32.626033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:32.627059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:32.627059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:32.627059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:32.627059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:32.627059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:32.627059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:32.627059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:32.627059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:32.627059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:32.627059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:32.627059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:32.627059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:32.634159 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 275/562] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:32.636474 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf11d5-a4eb-4f72-9cf6-360e3fb73618 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:32.636889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf11d5-a4eb-4f72-9cf6-360e3fb73618 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:32.637251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf11d5-a4eb-4f72-9cf6-360e3fb73618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:32.666299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-45cf11d5-a4eb-4f72-9cf6-360e3fb73618 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:32.668179 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-45cf11d5-a4eb-4f72-9cf6-360e3fb73618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:32.715563 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:32.731017 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d330e5-6ad0-435a-9dbd-f566c961f199 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:32.769494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 276/563] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:32.769494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf11d5-a4eb-4f72-9cf6-360e3fb73618 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:32.781954 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:32.805429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cee27-6535-4dd0-8242-5a212df352c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:32.806388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cee27-6535-4dd0-8242-5a212df352c8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:32.806850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cee27-6535-4dd0-8242-5a212df352c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:32.807306 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cee27-6535-4dd0-8242-5a212df352c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:32.808547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cee27-6535-4dd0-8242-5a212df352c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:32.809058 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cee27-6535-4dd0-8242-5a212df352c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:32.811571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cee27-6535-4dd0-8242-5a212df352c8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:32.835138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 288/564] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:32.836639 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cee27-6535-4dd0-8242-5a212df352c8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:32.903624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf11d5-a4eb-4f72-9cf6-360e3fb73618 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:32.953374 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d59cc1-667e-47a6-b825-a362e7f7e9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:32.954052 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d59cc1-667e-47a6-b825-a362e7f7e9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:32.954352 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d59cc1-667e-47a6-b825-a362e7f7e9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:32.954763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d59cc1-667e-47a6-b825-a362e7f7e9c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:32.956018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d59cc1-667e-47a6-b825-a362e7f7e9c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:32.956353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d59cc1-667e-47a6-b825-a362e7f7e9c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:32.957909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d59cc1-667e-47a6-b825-a362e7f7e9c8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:32.960191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cee27-6535-4dd0-8242-5a212df352c8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:32.972043 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 277/565] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:32.973006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d59cc1-667e-47a6-b825-a362e7f7e9c8 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:33.016898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f219b-ef7e-4ec0-add3-dda81b2ac085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:33.019411 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f219b-ef7e-4ec0-add3-dda81b2ac085 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:33.019902 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f219b-ef7e-4ec0-add3-dda81b2ac085 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:33.020406 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f219b-ef7e-4ec0-add3-dda81b2ac085 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:33.021797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f219b-ef7e-4ec0-add3-dda81b2ac085 None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:33.022318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f219b-ef7e-4ec0-add3-dda81b2ac085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:33.024940 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f219b-ef7e-4ec0-add3-dda81b2ac085 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:33.041977 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 289/566] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:33.041977 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f219b-ef7e-4ec0-add3-dda81b2ac085 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:33.063592 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:33.069147 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:33.123327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d59cc1-667e-47a6-b825-a362e7f7e9c8 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:33.174096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27e560-9eb7-4f60-92ff-46b097d6bbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:33.175066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27e560-9eb7-4f60-92ff-46b097d6bbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:33.175523 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27e560-9eb7-4f60-92ff-46b097d6bbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:33.175725 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f219b-ef7e-4ec0-add3-dda81b2ac085 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:33.176140 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27e560-9eb7-4f60-92ff-46b097d6bbd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:33.188288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb27e560-9eb7-4f60-92ff-46b097d6bbd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-fb27e560-9eb7-4f60-92ff-46b097d6bbd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:33.194343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:33.195435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:33.196458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:33.196458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:33.196458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:33.196458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:33.196458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:33.196458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:33.196458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:33.196458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:33.196458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:33.196458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:33.196458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:33.196458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:33.198616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 278/567] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:33.199773 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27e560-9eb7-4f60-92ff-46b097d6bbd8 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:33.241501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932061c0-0f9a-43c3-8732-006bbf9b24a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:33.242625 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932061c0-0f9a-43c3-8732-006bbf9b24a1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:33.243110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932061c0-0f9a-43c3-8732-006bbf9b24a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:33.243603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932061c0-0f9a-43c3-8732-006bbf9b24a1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:33.256897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932061c0-0f9a-43c3-8732-006bbf9b24a1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:33.259395 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932061c0-0f9a-43c3-8732-006bbf9b24a1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-932061c0-0f9a-43c3-8732-006bbf9b24a1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:33.282174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:33.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:33.290504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:33.290504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:33.290504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:33.290504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:33.290504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:33.290504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:33.290504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:33.290504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:33.290504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 10:30:33.290504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:33.290504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 290/568] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:33.290504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932061c0-0f9a-43c3-8732-006bbf9b24a1 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:33.345892 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27e560-9eb7-4f60-92ff-46b097d6bbd8 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:33.384951 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:33.389299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:33.398096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42683a-52ce-404c-9ccb-e82d7e8b0492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:33.398840 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42683a-52ce-404c-9ccb-e82d7e8b0492 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:33.399121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42683a-52ce-404c-9ccb-e82d7e8b0492 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:33.399413 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42683a-52ce-404c-9ccb-e82d7e8b0492 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:33.400236 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42683a-52ce-404c-9ccb-e82d7e8b0492 None admin] RBAC: Adding query filter params (role.id=1df33ad923a44a59b50dcacb6a75fefb, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=a81ea5afe74b43879abc94b3cd134fd2) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:33.400547 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42683a-52ce-404c-9ccb-e82d7e8b0492 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:33.402170 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42683a-52ce-404c-9ccb-e82d7e8b0492 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:33.411017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 279/569] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:30:33 2022] GET /identity/v3/role_assignments?user.id=a81ea5afe74b43879abc94b3cd134fd2&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=1df33ad923a44a59b50dcacb6a75fefb => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:33.411482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42683a-52ce-404c-9ccb-e82d7e8b0492 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:33.450738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932061c0-0f9a-43c3-8732-006bbf9b24a1 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:33.527439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461b1be-6218-4da3-a4c6-6a30c454f46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:33.528860 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461b1be-6218-4da3-a4c6-6a30c454f46d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:33.529426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461b1be-6218-4da3-a4c6-6a30c454f46d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:33.530078 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461b1be-6218-4da3-a4c6-6a30c454f46d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:33.539588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42683a-52ce-404c-9ccb-e82d7e8b0492 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:33.552011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461b1be-6218-4da3-a4c6-6a30c454f46d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:33.555124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461b1be-6218-4da3-a4c6-6a30c454f46d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:33.584349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a22599-9b22-4660-8ec4-ba89a44042ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:33.585286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a22599-9b22-4660-8ec4-ba89a44042ec None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:33.585758 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a22599-9b22-4660-8ec4-ba89a44042ec None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:33.586257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a22599-9b22-4660-8ec4-ba89a44042ec None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:33.587988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a22599-9b22-4660-8ec4-ba89a44042ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:33.588563 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a22599-9b22-4660-8ec4-ba89a44042ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:33.590642 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a22599-9b22-4660-8ec4-ba89a44042ec None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:33.594322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-1461b1be-6218-4da3-a4c6-6a30c454f46d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:33.594322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:33.594322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:33.594322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:33.594322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:33.594322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:33.594322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:33.594322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:33.594322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:33.594322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:33.594322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:33.594322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:33.594322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:33.594322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:33.594322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:33.595225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:33.596177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:33.596177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:33.596177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:33.596177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:33.597164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 291/570] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:33.598711 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461b1be-6218-4da3-a4c6-6a30c454f46d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:33.610848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 280/571] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:33.610848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a22599-9b22-4660-8ec4-ba89a44042ec None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:33.697799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:33.787096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a22599-9b22-4660-8ec4-ba89a44042ec None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:33.792855 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461b1be-6218-4da3-a4c6-6a30c454f46d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:33.847395 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:33.857608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15faba5d-2c02-4794-99ff-6ae65f642c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:33.858593 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faba5d-2c02-4794-99ff-6ae65f642c7b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:33.859102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faba5d-2c02-4794-99ff-6ae65f642c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:33.859547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faba5d-2c02-4794-99ff-6ae65f642c7b None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:33.860851 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149edc6-aca2-4fc0-9ba3-f14e8bb8ccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:33.861047 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15faba5d-2c02-4794-99ff-6ae65f642c7b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:33.861455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15faba5d-2c02-4794-99ff-6ae65f642c7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:33.862708 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149edc6-aca2-4fc0-9ba3-f14e8bb8ccf4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:33.863465 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149edc6-aca2-4fc0-9ba3-f14e8bb8ccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:33.864101 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15faba5d-2c02-4794-99ff-6ae65f642c7b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:33.864450 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149edc6-aca2-4fc0-9ba3-f14e8bb8ccf4 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:33.865974 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149edc6-aca2-4fc0-9ba3-f14e8bb8ccf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:33.866755 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149edc6-aca2-4fc0-9ba3-f14e8bb8ccf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:33.869706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149edc6-aca2-4fc0-9ba3-f14e8bb8ccf4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:33.892747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 281/572] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:33.895197 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149edc6-aca2-4fc0-9ba3-f14e8bb8ccf4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:33.898447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 292/573] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Aug 3 10:30:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:33.899927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15faba5d-2c02-4794-99ff-6ae65f642c7b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:34.092701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149edc6-aca2-4fc0-9ba3-f14e8bb8ccf4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:34.097519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15faba5d-2c02-4794-99ff-6ae65f642c7b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:34.126830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:34.169286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d9deb-9f18-4ec0-8c43-2d0f7cd77d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:34.169286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d9deb-9f18-4ec0-8c43-2d0f7cd77d7c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:34.169605 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f60826-974f-48f8-9150-d022e6a845c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:34.169735 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d9deb-9f18-4ec0-8c43-2d0f7cd77d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:34.169735 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d9deb-9f18-4ec0-8c43-2d0f7cd77d7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:34.169866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f60826-974f-48f8-9150-d022e6a845c2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:34.169866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f60826-974f-48f8-9150-d022e6a845c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:34.169866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f60826-974f-48f8-9150-d022e6a845c2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:34.181166 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d9deb-9f18-4ec0-8c43-2d0f7cd77d7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:34.183222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d9deb-9f18-4ec0-8c43-2d0f7cd77d7c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:34.187491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f60826-974f-48f8-9150-d022e6a845c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:34.191797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f60826-974f-48f8-9150-d022e6a845c2 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:34.205271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-a12d9deb-9f18-4ec0-8c43-2d0f7cd77d7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:34.205271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:34.205271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:34.205271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:34.205271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:34.205271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:34.205271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:34.205271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:34.205271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:34.205271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:34.205271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:34.205271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:34.205271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:34.205271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:34.205271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:34.206852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:34.207777 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:34.207777 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:34.207777 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:34.207777 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:34.210903 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 282/574] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:34.212155 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d9deb-9f18-4ec0-8c43-2d0f7cd77d7c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-06f60826-974f-48f8-9150-d022e6a845c2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:34.223815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:34.225963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:34.227751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:34.227751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:34.227751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:34.227751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:34.227751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:34.227751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:34.227751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:34.227751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:34.227751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 03 10:30:34.227751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:34.230292 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 293/575] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:34.231483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f60826-974f-48f8-9150-d022e6a845c2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:34.323171 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:34.402017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d9deb-9f18-4ec0-8c43-2d0f7cd77d7c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:34.406778 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f60826-974f-48f8-9150-d022e6a845c2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:34.450558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f8d49-1587-4aa5-8c82-d6fcb90eebbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:34.451193 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f8d49-1587-4aa5-8c82-d6fcb90eebbb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:34.451472 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f8d49-1587-4aa5-8c82-d6fcb90eebbb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:34.451753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f8d49-1587-4aa5-8c82-d6fcb90eebbb None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:34.452476 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f8d49-1587-4aa5-8c82-d6fcb90eebbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:34.452783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f8d49-1587-4aa5-8c82-d6fcb90eebbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:34.454296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f8d49-1587-4aa5-8c82-d6fcb90eebbb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:34.460763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa2916-7673-4d18-9f50-85fea2df1797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:34.461888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa2916-7673-4d18-9f50-85fea2df1797 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:34.462385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa2916-7673-4d18-9f50-85fea2df1797 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:34.462889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa2916-7673-4d18-9f50-85fea2df1797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:34.476915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 294/576] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:34.476915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f8d49-1587-4aa5-8c82-d6fcb90eebbb None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:34.487980 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa2916-7673-4d18-9f50-85fea2df1797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:34.490843 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa2916-7673-4d18-9f50-85fea2df1797 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:34.519388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-edfa2916-7673-4d18-9f50-85fea2df1797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:34.519388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:34.519388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:34.519388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:34.519388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:34.519388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:34.519388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:34.519388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:34.519388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:34.519388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:34.519388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:34.519388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:34.519388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:34.519388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:34.519388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:34.520571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:34.521700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:34.521700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:34.521700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:34.521700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:34.528805 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 283/577] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:34.531151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed6e0d-b976-45f5-abe1-f8e3fbc13b1a None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:34.531620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed6e0d-b976-45f5-abe1-f8e3fbc13b1a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:34.532063 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed6e0d-b976-45f5-abe1-f8e3fbc13b1a None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:34.537511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 284/578] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:34.537511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed6e0d-b976-45f5-abe1-f8e3fbc13b1a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:34.586878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:34.617268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f8d49-1587-4aa5-8c82-d6fcb90eebbb None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:34.629787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:34.660259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ec0b2-6f61-4b6c-9d77-8befd042d2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:34.660907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ec0b2-6f61-4b6c-9d77-8befd042d2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:34.661285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ec0b2-6f61-4b6c-9d77-8befd042d2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:34.661652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ec0b2-6f61-4b6c-9d77-8befd042d2b7 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:34.662545 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ec0b2-6f61-4b6c-9d77-8befd042d2b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:34.662920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ec0b2-6f61-4b6c-9d77-8befd042d2b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:34.664784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ec0b2-6f61-4b6c-9d77-8befd042d2b7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:34.694292 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 295/579] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Aug 3 10:30:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:34.694292 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ec0b2-6f61-4b6c-9d77-8befd042d2b7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:34.727994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed6e0d-b976-45f5-abe1-f8e3fbc13b1a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:34.775586 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d84ba-a29a-42b6-abfd-4a4cd7415a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:34.776564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d84ba-a29a-42b6-abfd-4a4cd7415a9a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:34.776911 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d84ba-a29a-42b6-abfd-4a4cd7415a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:34.777251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d84ba-a29a-42b6-abfd-4a4cd7415a9a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:34.788039 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d84ba-a29a-42b6-abfd-4a4cd7415a9a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:34.789864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d84ba-a29a-42b6-abfd-4a4cd7415a9a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-006d84ba-a29a-42b6-abfd-4a4cd7415a9a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:34.811050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:34.812093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:34.813044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:34.813044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:34.813044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:34.813044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:34.813044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:34.813044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:34.813044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:34.813044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:34.813044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:30:34.813044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:34.820990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 285/580] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 3 10:30:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:34.822265 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fbb07-73f2-4d30-848b-75b4068bb2ad None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:34.822688 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fbb07-73f2-4d30-848b-75b4068bb2ad None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:34.823085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fbb07-73f2-4d30-848b-75b4068bb2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:34.844455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ec0b2-6f61-4b6c-9d77-8befd042d2b7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:34.851855 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-9e5fbb07-73f2-4d30-848b-75b4068bb2ad None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:34.853813 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9e5fbb07-73f2-4d30-848b-75b4068bb2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:34.908124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38b462-2800-4999-a77f-cf7ff5ba8d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:34.909838 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38b462-2800-4999-a77f-cf7ff5ba8d99 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:34.910405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38b462-2800-4999-a77f-cf7ff5ba8d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:34.910807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38b462-2800-4999-a77f-cf7ff5ba8d99 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:34.912049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38b462-2800-4999-a77f-cf7ff5ba8d99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:34.912545 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38b462-2800-4999-a77f-cf7ff5ba8d99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:34.914826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38b462-2800-4999-a77f-cf7ff5ba8d99 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:34.937672 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 296/581] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:34.938858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38b462-2800-4999-a77f-cf7ff5ba8d99 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:34.969899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 286/582] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:34.970677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fbb07-73f2-4d30-848b-75b4068bb2ad None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:34.977645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:34.995275 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:35.107170 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38b462-2800-4999-a77f-cf7ff5ba8d99 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:35.135433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fbb07-73f2-4d30-848b-75b4068bb2ad None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:35.155121 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1802835-a3ea-441d-8473-37276e3b568a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:35.156245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1802835-a3ea-441d-8473-37276e3b568a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:35.156732 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1802835-a3ea-441d-8473-37276e3b568a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:35.157346 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1802835-a3ea-441d-8473-37276e3b568a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:35.175181 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1802835-a3ea-441d-8473-37276e3b568a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:35.177089 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1802835-a3ea-441d-8473-37276e3b568a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:35.182241 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d44ae-f644-4b30-b4ee-fe7f4688cc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:35.182880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d44ae-f644-4b30-b4ee-fe7f4688cc77 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:35.183170 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d44ae-f644-4b30-b4ee-fe7f4688cc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:35.183447 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d44ae-f644-4b30-b4ee-fe7f4688cc77 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:35.184267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d44ae-f644-4b30-b4ee-fe7f4688cc77 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:35.184577 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d44ae-f644-4b30-b4ee-fe7f4688cc77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:35.186081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d44ae-f644-4b30-b4ee-fe7f4688cc77 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:35.196044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-d1802835-a3ea-441d-8473-37276e3b568a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:35.196044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:35.196044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:35.196044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:35.196044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:35.196044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:35.196044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:35.196044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:35.196044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:35.196044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:35.196044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:35.196044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:35.196044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:35.196044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:35.196044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:35.196991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:35.197927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:35.197927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:35.197927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:35.197927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:35.199997 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 297/583] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:35.201024 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1802835-a3ea-441d-8473-37276e3b568a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:35.212599 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 287/584] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 3 10:30:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:35.213496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d44ae-f644-4b30-b4ee-fe7f4688cc77 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:35.298624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:35.305807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:35.333360 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1802835-a3ea-441d-8473-37276e3b568a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:35.349918 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d44ae-f644-4b30-b4ee-fe7f4688cc77 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:35.393864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a7fdc-a314-40e6-9e39-6c9b4c32e041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:35.394534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a7fdc-a314-40e6-9e39-6c9b4c32e041 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:35.394819 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a7fdc-a314-40e6-9e39-6c9b4c32e041 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:35.395092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a7fdc-a314-40e6-9e39-6c9b4c32e041 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:35.407077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc672-b1b1-402c-9bff-adc45f00a348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:35.407555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a7fdc-a314-40e6-9e39-6c9b4c32e041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:35.410203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc672-b1b1-402c-9bff-adc45f00a348 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:35.410954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc672-b1b1-402c-9bff-adc45f00a348 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:35.411689 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc672-b1b1-402c-9bff-adc45f00a348 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-6d5a7fdc-a314-40e6-9e39-6c9b4c32e041 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:35.413107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:35.415046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:35.416878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:35.416878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:35.416878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:35.416878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:35.416878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:35.416878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:35.416878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:35.416878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:35.416878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:35.416878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:35.416878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:35.416878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:35.419305 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 288/585] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:35.420277 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a7fdc-a314-40e6-9e39-6c9b4c32e041 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:35.432107 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cc672-b1b1-402c-9bff-adc45f00a348 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:35.435290 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cc672-b1b1-402c-9bff-adc45f00a348 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:35.459134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-971cc672-b1b1-402c-9bff-adc45f00a348 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:35.459134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:35.459134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:35.459134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:35.459134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:35.459134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:35.459134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:35.459134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:35.459134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:35.459134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:35.459134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:35.459134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:35.459134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:35.459134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:35.459134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:35.460795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:35.462528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:35.462528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:35.462528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:35.462528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:35.462528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:35.462528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:35.462528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:35.462528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:35.465726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 298/586] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:35.466630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc672-b1b1-402c-9bff-adc45f00a348 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:35.567757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a7fdc-a314-40e6-9e39-6c9b4c32e041 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:35.612832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:35.624429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e2b87-bce8-424c-b10e-ac4823162cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:35.625250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e2b87-bce8-424c-b10e-ac4823162cab None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:35.625719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e2b87-bce8-424c-b10e-ac4823162cab None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:35.626127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e2b87-bce8-424c-b10e-ac4823162cab None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:35.627126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e2b87-bce8-424c-b10e-ac4823162cab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:35.628126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e2b87-bce8-424c-b10e-ac4823162cab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:35.630052 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e2b87-bce8-424c-b10e-ac4823162cab None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:35.651953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 289/587] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:35.653399 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e2b87-bce8-424c-b10e-ac4823162cab None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:35.661030 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc672-b1b1-402c-9bff-adc45f00a348 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:35.725846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db491e3-77ab-4697-abc7-287dfa151edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:35.726833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db491e3-77ab-4697-abc7-287dfa151edd None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:35.727413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db491e3-77ab-4697-abc7-287dfa151edd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:35.727986 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db491e3-77ab-4697-abc7-287dfa151edd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:35.738514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:35.747804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db491e3-77ab-4697-abc7-287dfa151edd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:35.749321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db491e3-77ab-4697-abc7-287dfa151edd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:35.780157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-6db491e3-77ab-4697-abc7-287dfa151edd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:35.780157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:35.780157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:35.780157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:35.780157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:35.780157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:35.780157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:35.780157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:35.780157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:35.780157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:35.780157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:35.780157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:35.780157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:35.780157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:35.780157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:35.782050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:35.783907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:35.783907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:35.783907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:35.783907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:35.784823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 299/588] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:35.786202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db491e3-77ab-4697-abc7-287dfa151edd None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:35.836198 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e2b87-bce8-424c-b10e-ac4823162cab None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:35.897359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f04ce5-be1e-46f0-9e43-bc22d5907295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:35.898497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f04ce5-be1e-46f0-9e43-bc22d5907295 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:35.899001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f04ce5-be1e-46f0-9e43-bc22d5907295 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:35.899551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f04ce5-be1e-46f0-9e43-bc22d5907295 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:35.900970 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f04ce5-be1e-46f0-9e43-bc22d5907295 None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:35.901627 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f04ce5-be1e-46f0-9e43-bc22d5907295 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:35.904175 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f04ce5-be1e-46f0-9e43-bc22d5907295 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:35.925697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 290/589] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:35.927408 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f04ce5-be1e-46f0-9e43-bc22d5907295 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:35.960533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db491e3-77ab-4697-abc7-287dfa151edd None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:35.997394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:36.017875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69319b11-3d91-4239-805d-a6967422f578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:36.018880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69319b11-3d91-4239-805d-a6967422f578 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:36.019303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69319b11-3d91-4239-805d-a6967422f578 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:36.019761 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69319b11-3d91-4239-805d-a6967422f578 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:36.021113 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69319b11-3d91-4239-805d-a6967422f578 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:36.021965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69319b11-3d91-4239-805d-a6967422f578 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:36.024276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69319b11-3d91-4239-805d-a6967422f578 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:36.039214 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 300/590] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:36.040087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69319b11-3d91-4239-805d-a6967422f578 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:36.070854 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f04ce5-be1e-46f0-9e43-bc22d5907295 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:36.102710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:36.116167 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ac760-5017-4d64-bfb6-0af4ce7e1760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:36.117006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ac760-5017-4d64-bfb6-0af4ce7e1760 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:36.117270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ac760-5017-4d64-bfb6-0af4ce7e1760 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:36.117811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ac760-5017-4d64-bfb6-0af4ce7e1760 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:36.135332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ac760-5017-4d64-bfb6-0af4ce7e1760 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:36.138522 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ac760-5017-4d64-bfb6-0af4ce7e1760 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:36.152244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-c47ac760-5017-4d64-bfb6-0af4ce7e1760 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:36.152244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:36.152244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:36.152244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:36.152244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:36.152244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:36.152244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:36.152244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:36.152244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:36.152244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:36.152244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:36.152244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:36.152244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:36.152244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:36.152244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:36.153831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:36.154879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:36.154879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:36.154879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:36.154879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:36.154879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:36.154879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:36.154879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:36.154879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:36.154879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 291/591] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:36.155436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ac760-5017-4d64-bfb6-0af4ce7e1760 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:36.178514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69319b11-3d91-4239-805d-a6967422f578 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:36.234948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40bd1f5-2739-4765-b40d-b875b970e603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:36.235709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bd1f5-2739-4765-b40d-b875b970e603 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:36.236155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bd1f5-2739-4765-b40d-b875b970e603 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:36.236521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bd1f5-2739-4765-b40d-b875b970e603 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:36.237926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40bd1f5-2739-4765-b40d-b875b970e603 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:36.238429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40bd1f5-2739-4765-b40d-b875b970e603 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:36.240352 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40bd1f5-2739-4765-b40d-b875b970e603 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:36.256586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 301/592] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:36.258822 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608f8af-91a0-4e03-ad0f-50c525f58347 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:36.259421 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608f8af-91a0-4e03-ad0f-50c525f58347 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:36.259794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608f8af-91a0-4e03-ad0f-50c525f58347 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:36.265558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 302/593] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:36.265558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608f8af-91a0-4e03-ad0f-50c525f58347 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:36.317406 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ac760-5017-4d64-bfb6-0af4ce7e1760 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:36.331658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:36.371061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e428c5-62e2-410b-ba52-68131c09de0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:36.372083 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e428c5-62e2-410b-ba52-68131c09de0f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:36.372563 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e428c5-62e2-410b-ba52-68131c09de0f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:36.373047 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e428c5-62e2-410b-ba52-68131c09de0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:36.390661 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e428c5-62e2-410b-ba52-68131c09de0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:36.392157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e428c5-62e2-410b-ba52-68131c09de0f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:36.405534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608f8af-91a0-4e03-ad0f-50c525f58347 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:36.413317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-e1e428c5-62e2-410b-ba52-68131c09de0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:36.413317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:36.413317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:36.413317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:36.413317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:36.413317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:36.413317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:36.413317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:36.413317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:36.413317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:36.413317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:36.413317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:36.413317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:36.413317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:36.413317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:36.416861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:36.418601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:36.418601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:36.418601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:36.418601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:36.418601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 292/594] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:36.418601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e428c5-62e2-410b-ba52-68131c09de0f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:36.422346 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:36.461657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d03365-040a-43f0-a5f3-68452d93b2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:36.462631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d03365-040a-43f0-a5f3-68452d93b2b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:36.463111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d03365-040a-43f0-a5f3-68452d93b2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:36.463509 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d03365-040a-43f0-a5f3-68452d93b2b4 None admin] PATH_INFO: `/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/3b5f547a388c4f64aeb95a7435818741/roles/1df33ad923a44a59b50dcacb6a75fefb` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:36.485388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d03365-040a-43f0-a5f3-68452d93b2b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=4272b859979d49c8b8f0c17ec9eb81bc, user_id=3b5f547a388c4f64aeb95a7435818741, role_id=1df33ad923a44a59b50dcacb6a75fefb)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:36.487595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d03365-040a-43f0-a5f3-68452d93b2b4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:36.503418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 303/595] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:30:36 2022] PUT /identity/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/3b5f547a388c4f64aeb95a7435818741/roles/1df33ad923a44a59b50dcacb6a75fefb => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:30:36.504568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d03365-040a-43f0-a5f3-68452d93b2b4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:36.593250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e428c5-62e2-410b-ba52-68131c09de0f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:36.681270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ae976-9f12-40a7-8f22-902a5785211c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:36.682256 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ae976-9f12-40a7-8f22-902a5785211c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:36.682717 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ae976-9f12-40a7-8f22-902a5785211c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:36.683198 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ae976-9f12-40a7-8f22-902a5785211c None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:36.684686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ae976-9f12-40a7-8f22-902a5785211c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:36.685341 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ae976-9f12-40a7-8f22-902a5785211c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:36.689535 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ae976-9f12-40a7-8f22-902a5785211c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:36.704314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d03365-040a-43f0-a5f3-68452d93b2b4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:36.709799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 293/596] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:36.716651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5387725-e989-4a6f-81e2-54b0a419815d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:36.717588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5387725-e989-4a6f-81e2-54b0a419815d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:36.718250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5387725-e989-4a6f-81e2-54b0a419815d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:36.755248 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d5387725-e989-4a6f-81e2-54b0a419815d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:36.758009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d5387725-e989-4a6f-81e2-54b0a419815d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:36.792978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0908c6e-3a3f-4ac0-a83f-c6c7565f19c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:36.794627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0908c6e-3a3f-4ac0-a83f-c6c7565f19c2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:36.796579 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:36.797428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0908c6e-3a3f-4ac0-a83f-c6c7565f19c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:36.798202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0908c6e-3a3f-4ac0-a83f-c6c7565f19c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:36.820571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0908c6e-3a3f-4ac0-a83f-c6c7565f19c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:36.824058 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0908c6e-3a3f-4ac0-a83f-c6c7565f19c2 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:36.850220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-f0908c6e-3a3f-4ac0-a83f-c6c7565f19c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:36.850220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:36.850220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:36.850220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:36.850220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:36.850220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:36.850220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:36.850220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:36.850220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:36.850220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:36.850220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:36.850220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:36.850220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:36.850220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:36.850220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:36.852212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:36.854367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:36.854367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:36.854367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:36.854367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:36.854367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:36.854367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:36.854367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:36.854367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:36.862797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 304/597] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:36.864479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0908c6e-3a3f-4ac0-a83f-c6c7565f19c2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:36.893356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 294/598] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:36.895138 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5387725-e989-4a6f-81e2-54b0a419815d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:36.938405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:37.125282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0908c6e-3a3f-4ac0-a83f-c6c7565f19c2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:37.161365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5387725-e989-4a6f-81e2-54b0a419815d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:37.212171 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699011c-9698-45f5-bf13-e22ad4ebb583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:37.213245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699011c-9698-45f5-bf13-e22ad4ebb583 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:37.214442 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699011c-9698-45f5-bf13-e22ad4ebb583 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:37.214925 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699011c-9698-45f5-bf13-e22ad4ebb583 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:37.216484 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699011c-9698-45f5-bf13-e22ad4ebb583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:37.217030 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699011c-9698-45f5-bf13-e22ad4ebb583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:37.219675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699011c-9698-45f5-bf13-e22ad4ebb583 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:37.222584 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e9b7a-23d2-4614-bc3f-07d0c6dd8c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:37.224071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e9b7a-23d2-4614-bc3f-07d0c6dd8c3c None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:37.225803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e9b7a-23d2-4614-bc3f-07d0c6dd8c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:37.226699 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e9b7a-23d2-4614-bc3f-07d0c6dd8c3c None admin] PATH_INFO: `/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/300fb5e1c5c546808b935871673e3d0f/roles/1df33ad923a44a59b50dcacb6a75fefb` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:37.249707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 305/599] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:37.251327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699011c-9698-45f5-bf13-e22ad4ebb583 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:37.261299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e9b7a-23d2-4614-bc3f-07d0c6dd8c3c None admin] RBAC: Authorizing `identity:create_grant(project_id=4272b859979d49c8b8f0c17ec9eb81bc, user_id=300fb5e1c5c546808b935871673e3d0f, role_id=1df33ad923a44a59b50dcacb6a75fefb)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:37.272150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e9b7a-23d2-4614-bc3f-07d0c6dd8c3c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:37.302827 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 295/600] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:30:36 2022] PUT /identity/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/300fb5e1c5c546808b935871673e3d0f/roles/1df33ad923a44a59b50dcacb6a75fefb => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:30:37.304585 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e9b7a-23d2-4614-bc3f-07d0c6dd8c3c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:37.309279 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:37.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:37.576431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699011c-9698-45f5-bf13-e22ad4ebb583 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:37.641172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4134506-c5dc-41de-8660-611166928c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:37.642052 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4134506-c5dc-41de-8660-611166928c01 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:37.642449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4134506-c5dc-41de-8660-611166928c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:37.642819 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4134506-c5dc-41de-8660-611166928c01 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:37.650599 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4134506-c5dc-41de-8660-611166928c01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:37.651194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4134506-c5dc-41de-8660-611166928c01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:37.652781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4134506-c5dc-41de-8660-611166928c01 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:37.661681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e9b7a-23d2-4614-bc3f-07d0c6dd8c3c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:37.673609 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 306/601] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:37.674829 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4134506-c5dc-41de-8660-611166928c01 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:37.743906 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86ded2-2032-4777-b2a2-faa5bf05b2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:37.749501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86ded2-2032-4777-b2a2-faa5bf05b2ae None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:37.749501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86ded2-2032-4777-b2a2-faa5bf05b2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:37.749501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86ded2-2032-4777-b2a2-faa5bf05b2ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:37.776203 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86ded2-2032-4777-b2a2-faa5bf05b2ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-af86ded2-2032-4777-b2a2-faa5bf05b2ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:37.789908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:37.791858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:37.793634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:37.793634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:37.793634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:37.793634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:37.793634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:37.793634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:37.793634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:37.793634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:37.793634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:37.793634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:37.793634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:37.793634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:37.801520 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 296/602] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 493 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:37.801520 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86ded2-2032-4777-b2a2-faa5bf05b2ae None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:37.905498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4134506-c5dc-41de-8660-611166928c01 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:37.965121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:37.983305 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129e0f2-1e67-4949-8ada-039d8c61eae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:37.984237 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129e0f2-1e67-4949-8ada-039d8c61eae6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:37.986458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129e0f2-1e67-4949-8ada-039d8c61eae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:37.987456 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129e0f2-1e67-4949-8ada-039d8c61eae6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:38.009501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129e0f2-1e67-4949-8ada-039d8c61eae6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:38.009501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129e0f2-1e67-4949-8ada-039d8c61eae6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:38.021510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-2129e0f2-1e67-4949-8ada-039d8c61eae6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:38.038984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:38.040967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:38.051839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:38.051839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:38.051839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:38.051839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:38.051839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:38.051839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:38.051839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:38.051839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:38.051839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 10:30:38.051839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:38.051839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 307/603] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:38.051839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129e0f2-1e67-4949-8ada-039d8c61eae6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:38.067912 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86ded2-2032-4777-b2a2-faa5bf05b2ae None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:38.149391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac85245-70ef-479f-9f78-c92d840d11b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:38.151017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac85245-70ef-479f-9f78-c92d840d11b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:38.152154 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac85245-70ef-479f-9f78-c92d840d11b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:38.152901 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac85245-70ef-479f-9f78-c92d840d11b1 None admin] PATH_INFO: `/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/e66f25cc329f418eba6bd637bf3e8a3a/roles/1df33ad923a44a59b50dcacb6a75fefb` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:38.188451 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac85245-70ef-479f-9f78-c92d840d11b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4272b859979d49c8b8f0c17ec9eb81bc, user_id=e66f25cc329f418eba6bd637bf3e8a3a, role_id=1df33ad923a44a59b50dcacb6a75fefb)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:38.191478 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac85245-70ef-479f-9f78-c92d840d11b1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:38.239765 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 297/604] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:30:37 2022] PUT /identity/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/e66f25cc329f418eba6bd637bf3e8a3a/roles/1df33ad923a44a59b50dcacb6a75fefb => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:30:38.240633 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac85245-70ef-479f-9f78-c92d840d11b1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:38.384526 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129e0f2-1e67-4949-8ada-039d8c61eae6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:38.524126 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf7af0-5470-4e8b-ad46-087351ab7b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:38.525760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf7af0-5470-4e8b-ad46-087351ab7b6d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:38.526531 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf7af0-5470-4e8b-ad46-087351ab7b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:38.527322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf7af0-5470-4e8b-ad46-087351ab7b6d None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:38.528891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf7af0-5470-4e8b-ad46-087351ab7b6d None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:38.532657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf7af0-5470-4e8b-ad46-087351ab7b6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:38.536155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf7af0-5470-4e8b-ad46-087351ab7b6d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:38.574053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 308/605] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 526 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:38.592460 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf7af0-5470-4e8b-ad46-087351ab7b6d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:38.628084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac85245-70ef-479f-9f78-c92d840d11b1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:38.669514 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:38.774535 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ba1d1-05a4-4bac-837b-45fe706ae492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:38.775555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ba1d1-05a4-4bac-837b-45fe706ae492 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:38.776006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ba1d1-05a4-4bac-837b-45fe706ae492 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:38.776465 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ba1d1-05a4-4bac-837b-45fe706ae492 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:38.778920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ba1d1-05a4-4bac-837b-45fe706ae492 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:38.779260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ba1d1-05a4-4bac-837b-45fe706ae492 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:38.781964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ba1d1-05a4-4bac-837b-45fe706ae492 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:38.791997 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:38.865266 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 298/606] 10.222.0.47 () {56 vars in 1139 bytes} [Wed Aug 3 10:30:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 626 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:38.872052 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ba1d1-05a4-4bac-837b-45fe706ae492 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:38.952374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf7af0-5470-4e8b-ad46-087351ab7b6d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:39.080283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec623c20-a7c2-45c2-b553-58cb3e13af42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:39.081623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec623c20-a7c2-45c2-b553-58cb3e13af42 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:39.085523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec623c20-a7c2-45c2-b553-58cb3e13af42 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:39.085523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec623c20-a7c2-45c2-b553-58cb3e13af42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:39.111312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec623c20-a7c2-45c2-b553-58cb3e13af42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:39.114278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec623c20-a7c2-45c2-b553-58cb3e13af42 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:39.156164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ec623c20-a7c2-45c2-b553-58cb3e13af42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:39.156164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:39.156164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:39.156164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:39.156164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:39.156164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:39.156164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:39.156164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:39.156164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:39.156164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:39.156164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:39.156164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:39.156164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:39.156164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:39.156164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:39.157131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:39.158041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:39.158041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:39.158041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:39.158041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:39.158833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 309/607] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 567 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:30:39.160115 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9eb4fc-43b6-4e13-9316-6390bfb2a950 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:39.160543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9eb4fc-43b6-4e13-9316-6390bfb2a950 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:39.161020 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9eb4fc-43b6-4e13-9316-6390bfb2a950 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:39.169164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 310/608] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:39 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:39.170070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9eb4fc-43b6-4e13-9316-6390bfb2a950 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:39.206248 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ba1d1-05a4-4bac-837b-45fe706ae492 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:39.322055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c616df-044b-4c5a-a6d1-a9002e41c845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:39.322756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c616df-044b-4c5a-a6d1-a9002e41c845 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:39.325166 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c616df-044b-4c5a-a6d1-a9002e41c845 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:39.325166 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c616df-044b-4c5a-a6d1-a9002e41c845 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:39.350888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c616df-044b-4c5a-a6d1-a9002e41c845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:39.354725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c616df-044b-4c5a-a6d1-a9002e41c845 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:39.407936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-c9c616df-044b-4c5a-a6d1-a9002e41c845 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:39.407936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:39.407936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:39.407936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:39.407936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:39.407936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:39.407936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:39.407936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:39.407936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:39.407936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:39.407936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:39.407936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:39.407936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:39.407936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:39.407936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:39.409452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:39.410457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:39.410457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:39.410457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:39.410457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:39.415514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9eb4fc-43b6-4e13-9316-6390bfb2a950 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:39.422100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 299/609] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 551 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:30:39.425261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad98f1-9698-4f10-947a-eb5d93b0ab1d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:39.425668 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad98f1-9698-4f10-947a-eb5d93b0ab1d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:39.426150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad98f1-9698-4f10-947a-eb5d93b0ab1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:39.441651 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:39.468484 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-91ad98f1-9698-4f10-947a-eb5d93b0ab1d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:39.471707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad132c-3119-4d0b-8f6b-5e9f20c60177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:39.472936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad132c-3119-4d0b-8f6b-5e9f20c60177 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:39.473069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-91ad98f1-9698-4f10-947a-eb5d93b0ab1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:39.473610 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad132c-3119-4d0b-8f6b-5e9f20c60177 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:39.474087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad132c-3119-4d0b-8f6b-5e9f20c60177 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:39.475089 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad132c-3119-4d0b-8f6b-5e9f20c60177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:39.475672 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad132c-3119-4d0b-8f6b-5e9f20c60177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:39.479420 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad132c-3119-4d0b-8f6b-5e9f20c60177 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:39.499961 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 311/610] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:39.506810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad132c-3119-4d0b-8f6b-5e9f20c60177 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:39.520622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:39.651172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad132c-3119-4d0b-8f6b-5e9f20c60177 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:39.678037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 300/611] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:39.682502 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad98f1-9698-4f10-947a-eb5d93b0ab1d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:39.708130 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d33bf2-accd-478d-9d51-bca1ada988b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:39.709148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d33bf2-accd-478d-9d51-bca1ada988b9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:39.709700 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d33bf2-accd-478d-9d51-bca1ada988b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:39.710227 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d33bf2-accd-478d-9d51-bca1ada988b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:39.711198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d33bf2-accd-478d-9d51-bca1ada988b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:39.711774 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d33bf2-accd-478d-9d51-bca1ada988b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:39.713501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d33bf2-accd-478d-9d51-bca1ada988b9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:39.739629 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 312/612] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:39.740831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d33bf2-accd-478d-9d51-bca1ada988b9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:39.783018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:39.878619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad98f1-9698-4f10-947a-eb5d93b0ab1d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:39.902008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d33bf2-accd-478d-9d51-bca1ada988b9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:39.930570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042ae7f-7f2c-4fbb-ba12-4a99fb3d6e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:39.930570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042ae7f-7f2c-4fbb-ba12-4a99fb3d6e45 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:39.931531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042ae7f-7f2c-4fbb-ba12-4a99fb3d6e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:39.931807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042ae7f-7f2c-4fbb-ba12-4a99fb3d6e45 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:39.941789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042ae7f-7f2c-4fbb-ba12-4a99fb3d6e45 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:39.944076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042ae7f-7f2c-4fbb-ba12-4a99fb3d6e45 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:39.962246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345a55a-fdad-4444-9c4b-43d596e964c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:39.963181 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345a55a-fdad-4444-9c4b-43d596e964c4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-1042ae7f-7f2c-4fbb-ba12-4a99fb3d6e45 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:39.963316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:39.964473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:39.965380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:39.965380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:39.965380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:39.965380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:39.965380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:39.965380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:39.965380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:39.965380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:39.965380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 10:30:39.965380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:39.966792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345a55a-fdad-4444-9c4b-43d596e964c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:39.967262 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345a55a-fdad-4444-9c4b-43d596e964c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:39.969801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 301/613] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:30:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:39.972300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6da3df-1170-4169-a0ef-2141dd6b2aa2 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:39.972615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6da3df-1170-4169-a0ef-2141dd6b2aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:39.972893 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6da3df-1170-4169-a0ef-2141dd6b2aa2 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:39.975069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 302/614] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:39.975870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6da3df-1170-4169-a0ef-2141dd6b2aa2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:39.986467 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345a55a-fdad-4444-9c4b-43d596e964c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:39.994215 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-6345a55a-fdad-4444-9c4b-43d596e964c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:39.996483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:39.997595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:39.998431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:39.998431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:39.998431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:39.998431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:39.998431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:39.998431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:39.998431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:39.998431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:39.998431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:39.998431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:39.998431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:39.998431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:40.001515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 313/615] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:40.001515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345a55a-fdad-4444-9c4b-43d596e964c4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:40.143278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:40.151840 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6da3df-1170-4169-a0ef-2141dd6b2aa2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:40.154689 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345a55a-fdad-4444-9c4b-43d596e964c4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:40.200126 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd93e5-db53-4721-a0c9-1a270bfe82d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:40.201350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd93e5-db53-4721-a0c9-1a270bfe82d5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:40.201920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd93e5-db53-4721-a0c9-1a270bfe82d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:40.202474 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd93e5-db53-4721-a0c9-1a270bfe82d5 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:40.203549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd93e5-db53-4721-a0c9-1a270bfe82d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:40.204099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd93e5-db53-4721-a0c9-1a270bfe82d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:40.206676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd93e5-db53-4721-a0c9-1a270bfe82d5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:40.223211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bac24-a593-447a-bf80-46eb4e86a0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:40.224154 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bac24-a593-447a-bf80-46eb4e86a0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:40.224603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bac24-a593-447a-bf80-46eb4e86a0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:40.225069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bac24-a593-447a-bf80-46eb4e86a0f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:40.238442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bac24-a593-447a-bf80-46eb4e86a0f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:40.241034 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bac24-a593-447a-bf80-46eb4e86a0f4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:40.245638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 314/616] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Aug 3 10:30:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:40.246119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e20b1d-e5eb-4b8a-92cb-cb8a2ede8637 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:40.247606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e20b1d-e5eb-4b8a-92cb-cb8a2ede8637 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:40.247606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e20b1d-e5eb-4b8a-92cb-cb8a2ede8637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:40.262379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-347bac24-a593-447a-bf80-46eb4e86a0f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:40.262379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:40.262379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:40.262379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:40.262379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:40.262379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:40.262379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:40.262379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:40.262379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:40.262379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:40.262379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:40.262379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:40.262379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:40.262379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:40.262379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:40.263956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:40.265902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:40.265902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:40.265902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:40.265902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:40.265902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:40.265902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:40.265902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:40.265902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:40.267044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 303/617] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:40.267860 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bac24-a593-447a-bf80-46eb4e86a0f4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:40.292676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-82e20b1d-e5eb-4b8a-92cb-cb8a2ede8637 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:40.294849 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-82e20b1d-e5eb-4b8a-92cb-cb8a2ede8637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:40.369553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:40.423672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bac24-a593-447a-bf80-46eb4e86a0f4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:40.462028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 315/618] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:40.463769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e20b1d-e5eb-4b8a-92cb-cb8a2ede8637 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:40.487556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8278d-25a7-4acd-b29a-11401daf5ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:40.489553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8278d-25a7-4acd-b29a-11401daf5ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:40.490567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8278d-25a7-4acd-b29a-11401daf5ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:40.491172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8278d-25a7-4acd-b29a-11401daf5ad6 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:40.492656 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8278d-25a7-4acd-b29a-11401daf5ad6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:40.494188 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8278d-25a7-4acd-b29a-11401daf5ad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:40.496750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8278d-25a7-4acd-b29a-11401daf5ad6 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:40.513685 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 304/619] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:40.520434 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8278d-25a7-4acd-b29a-11401daf5ad6 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:40.598781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:40.637066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e20b1d-e5eb-4b8a-92cb-cb8a2ede8637 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:40.725781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88110701-9676-49b6-b671-860ac1a9a4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:40.727349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88110701-9676-49b6-b671-860ac1a9a4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:40.727930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88110701-9676-49b6-b671-860ac1a9a4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:40.728507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88110701-9676-49b6-b671-860ac1a9a4c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:40.750391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88110701-9676-49b6-b671-860ac1a9a4c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:40.753715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88110701-9676-49b6-b671-860ac1a9a4c4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:40.754664 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8278d-25a7-4acd-b29a-11401daf5ad6 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:40.787758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-88110701-9676-49b6-b671-860ac1a9a4c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:40.787758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:40.787758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:40.787758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:40.787758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:40.787758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:40.787758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:40.787758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:40.787758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:40.787758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:40.787758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:40.787758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:40.787758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:40.787758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:40.787758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:40.789917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:40.791985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:40.791985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:40.791985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:40.791985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:40.791985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 316/620] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:40.792605 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88110701-9676-49b6-b671-860ac1a9a4c4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:40.828193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13a55b-dc8e-401d-ab24-047c9a39e7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:40.829824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13a55b-dc8e-401d-ab24-047c9a39e7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:40.830162 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13a55b-dc8e-401d-ab24-047c9a39e7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:40.830989 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13a55b-dc8e-401d-ab24-047c9a39e7f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:40.831797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13a55b-dc8e-401d-ab24-047c9a39e7f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:40.832103 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13a55b-dc8e-401d-ab24-047c9a39e7f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:40.833872 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13a55b-dc8e-401d-ab24-047c9a39e7f3 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:40.835495 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:40.850022 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 305/621] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:40.851245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13a55b-dc8e-401d-ab24-047c9a39e7f3 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:40.947321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88110701-9676-49b6-b671-860ac1a9a4c4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:41.013913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7368c-36b9-46f4-9b11-ec1268fe5a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:41.014890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7368c-36b9-46f4-9b11-ec1268fe5a7a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:41.015622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7368c-36b9-46f4-9b11-ec1268fe5a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:41.015622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7368c-36b9-46f4-9b11-ec1268fe5a7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:41.019457 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:41.026067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13a55b-dc8e-401d-ab24-047c9a39e7f3 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:41.030073 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7368c-36b9-46f4-9b11-ec1268fe5a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-4ab7368c-36b9-46f4-9b11-ec1268fe5a7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:41.035863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:41.040377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:41.042756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:41.042756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:41.042756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:41.042756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:41.042756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:41.042756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:41.042756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:41.042756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:41.042756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:41.042756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:41.042756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:41.042756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:41.042756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 317/622] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:41.042756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adeac7-300e-4a7f-8ccd-326ba7bb9105 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:41.044364 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adeac7-300e-4a7f-8ccd-326ba7bb9105 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:41.044364 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adeac7-300e-4a7f-8ccd-326ba7bb9105 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:41.044364 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 318/623] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:41.044364 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5adeac7-300e-4a7f-8ccd-326ba7bb9105 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:41.079589 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e695c55-e9d3-4db3-b2fa-47a4b268d0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:41.081255 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e695c55-e9d3-4db3-b2fa-47a4b268d0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:41.081784 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e695c55-e9d3-4db3-b2fa-47a4b268d0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:41.082261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e695c55-e9d3-4db3-b2fa-47a4b268d0c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:41.094208 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e695c55-e9d3-4db3-b2fa-47a4b268d0c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:41.095680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e695c55-e9d3-4db3-b2fa-47a4b268d0c7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:41.114488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-4e695c55-e9d3-4db3-b2fa-47a4b268d0c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:41.114488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:41.114488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:41.114488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:41.114488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:41.114488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:41.114488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:41.114488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:41.114488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:41.114488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:41.114488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:41.114488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:41.114488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:41.114488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:41.114488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:41.116195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:41.117990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:41.117990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:41.117990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:41.117990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:41.117990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 306/624] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:41.117990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e695c55-e9d3-4db3-b2fa-47a4b268d0c7 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:41.184150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:41.218756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5adeac7-300e-4a7f-8ccd-326ba7bb9105 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:41.272462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f971b32-aeea-4899-be16-03f026a67646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:41.273009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f971b32-aeea-4899-be16-03f026a67646 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:41.273293 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f971b32-aeea-4899-be16-03f026a67646 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:41.273713 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f971b32-aeea-4899-be16-03f026a67646 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:41.274328 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f971b32-aeea-4899-be16-03f026a67646 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:41.274475 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e695c55-e9d3-4db3-b2fa-47a4b268d0c7 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:41.274669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f971b32-aeea-4899-be16-03f026a67646 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:41.276062 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f971b32-aeea-4899-be16-03f026a67646 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:41.297948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 319/625] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:41.299788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f971b32-aeea-4899-be16-03f026a67646 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:41.323568 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9047ab-e16c-423e-9f3b-1aa8545182de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:41.324262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9047ab-e16c-423e-9f3b-1aa8545182de None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:41.324685 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9047ab-e16c-423e-9f3b-1aa8545182de None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:41.324835 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9047ab-e16c-423e-9f3b-1aa8545182de None admin] PATH_INFO: `/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/1c211cc19a2045bd9383d31c7e81ceac/roles/1df33ad923a44a59b50dcacb6a75fefb` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:41.343254 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9047ab-e16c-423e-9f3b-1aa8545182de None admin] RBAC: Authorizing `identity:create_grant(project_id=4272b859979d49c8b8f0c17ec9eb81bc, user_id=1c211cc19a2045bd9383d31c7e81ceac, role_id=1df33ad923a44a59b50dcacb6a75fefb)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:41.344777 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9047ab-e16c-423e-9f3b-1aa8545182de None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:41.363191 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 307/626] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:30:41 2022] PUT /identity/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/1c211cc19a2045bd9383d31c7e81ceac/roles/1df33ad923a44a59b50dcacb6a75fefb => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:30:41.364757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdcf63-0b3e-4c5a-a70e-9ea925830047 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:41.365045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdcf63-0b3e-4c5a-a70e-9ea925830047 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:41.365323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdcf63-0b3e-4c5a-a70e-9ea925830047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:41.388418 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4ccdcf63-0b3e-4c5a-a70e-9ea925830047 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:41.390324 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4ccdcf63-0b3e-4c5a-a70e-9ea925830047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:41.409383 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:41.502307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f971b32-aeea-4899-be16-03f026a67646 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:41.517708 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 308/627] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:41.519368 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccdcf63-0b3e-4c5a-a70e-9ea925830047 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:41.574839 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:41.596591 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60bc6f-2f74-4704-844a-167a1d4856fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:41.597669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60bc6f-2f74-4704-844a-167a1d4856fb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:41.598169 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60bc6f-2f74-4704-844a-167a1d4856fb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:41.598660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60bc6f-2f74-4704-844a-167a1d4856fb None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:41.600147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60bc6f-2f74-4704-844a-167a1d4856fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:41.600682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60bc6f-2f74-4704-844a-167a1d4856fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:41.603202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60bc6f-2f74-4704-844a-167a1d4856fb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:41.621911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 320/628] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:41.622895 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60bc6f-2f74-4704-844a-167a1d4856fb None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:41.690307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccdcf63-0b3e-4c5a-a70e-9ea925830047 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:41.740553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae47fec-9ed4-45bc-9ffc-8f08f67b156c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:41.741497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae47fec-9ed4-45bc-9ffc-8f08f67b156c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:41.742086 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae47fec-9ed4-45bc-9ffc-8f08f67b156c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:41.742518 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae47fec-9ed4-45bc-9ffc-8f08f67b156c None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:41.743556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae47fec-9ed4-45bc-9ffc-8f08f67b156c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:41.743873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae47fec-9ed4-45bc-9ffc-8f08f67b156c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:41.745295 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae47fec-9ed4-45bc-9ffc-8f08f67b156c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:41.766050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 309/629] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:41.767068 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae47fec-9ed4-45bc-9ffc-8f08f67b156c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:41.786273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60bc6f-2f74-4704-844a-167a1d4856fb None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:41.810621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:41.837512 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8361450-1993-4e9b-8e62-b17b76025940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:41.838480 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8361450-1993-4e9b-8e62-b17b76025940 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:41.839219 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8361450-1993-4e9b-8e62-b17b76025940 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:41.839690 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8361450-1993-4e9b-8e62-b17b76025940 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:41.854189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8361450-1993-4e9b-8e62-b17b76025940 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:41.856718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8361450-1993-4e9b-8e62-b17b76025940 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:41.880975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-c8361450-1993-4e9b-8e62-b17b76025940 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:41.880975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:41.880975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:41.880975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:41.880975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:41.880975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:41.880975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:41.880975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:41.880975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:41.880975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:41.880975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:41.880975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:41.880975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:41.880975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:41.880975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:41.882124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:41.883090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:41.883090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:41.883090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:41.883090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:41.883090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:41.883090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:41.883090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:41.883090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:41.888771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 321/630] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:41.888771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8361450-1993-4e9b-8e62-b17b76025940 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:41.921006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:41.942009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae47fec-9ed4-45bc-9ffc-8f08f67b156c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:42.000894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0b0f0-ab8d-4a4b-aa01-00605834a22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:42.003491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0b0f0-ab8d-4a4b-aa01-00605834a22d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:42.004091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0b0f0-ab8d-4a4b-aa01-00605834a22d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:42.004861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0b0f0-ab8d-4a4b-aa01-00605834a22d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:42.027558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0b0f0-ab8d-4a4b-aa01-00605834a22d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-05d0b0f0-ab8d-4a4b-aa01-00605834a22d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:42.039192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:42.040199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:42.041069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:42.041069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:42.041069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:42.041069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:42.041069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:42.041069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:42.041069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:42.041069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:42.041069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:42.041069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:42.041069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:42.041069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:42.042443 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 310/631] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:42.044097 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0b0f0-ab8d-4a4b-aa01-00605834a22d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:42.107446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8361450-1993-4e9b-8e62-b17b76025940 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:42.169257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8284dc2-9601-4d43-9395-e5922bb450a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:42.170242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8284dc2-9601-4d43-9395-e5922bb450a9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:42.170701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8284dc2-9601-4d43-9395-e5922bb450a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:42.171131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8284dc2-9601-4d43-9395-e5922bb450a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:42.200554 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8284dc2-9601-4d43-9395-e5922bb450a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:42.204761 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8284dc2-9601-4d43-9395-e5922bb450a9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:42.220116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0b0f0-ab8d-4a4b-aa01-00605834a22d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:42.233680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-d8284dc2-9601-4d43-9395-e5922bb450a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:42.233680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:42.233680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:42.233680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:42.233680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:42.233680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:42.233680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:42.233680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:42.233680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:42.233680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:42.233680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:42.233680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:42.233680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:42.233680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:42.233680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:42.234661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:42.235555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:42.235555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:42.235555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:42.235555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:42.238113 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 322/632] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:42.239460 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8284dc2-9601-4d43-9395-e5922bb450a9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:42.281142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b463f-1dd1-4bbf-b6e6-5eb9e461880f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:42.282622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b463f-1dd1-4bbf-b6e6-5eb9e461880f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:42.282859 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b463f-1dd1-4bbf-b6e6-5eb9e461880f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:42.283127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b463f-1dd1-4bbf-b6e6-5eb9e461880f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:42.290496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:42.295822 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b463f-1dd1-4bbf-b6e6-5eb9e461880f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:42.298607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b463f-1dd1-4bbf-b6e6-5eb9e461880f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-001b463f-1dd1-4bbf-b6e6-5eb9e461880f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:42.324078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:42.325941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:42.328320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:42.328320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:42.328320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:42.328320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:42.328320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:42.328320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:42.328320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:42.328320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:42.328320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 10:30:42.328320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:42.333517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 311/633] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:42.335173 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b463f-1dd1-4bbf-b6e6-5eb9e461880f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:42.354438 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:42.451487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8284dc2-9601-4d43-9395-e5922bb450a9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:42.526443 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff025be-3bb0-4a34-9170-49796875127b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:42.528531 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff025be-3bb0-4a34-9170-49796875127b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:42.529602 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff025be-3bb0-4a34-9170-49796875127b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:42.530339 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff025be-3bb0-4a34-9170-49796875127b None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:42.532091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff025be-3bb0-4a34-9170-49796875127b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:42.532708 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff025be-3bb0-4a34-9170-49796875127b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:42.533381 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b463f-1dd1-4bbf-b6e6-5eb9e461880f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:42.535042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff025be-3bb0-4a34-9170-49796875127b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:42.553161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 323/634] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:42.554139 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff025be-3bb0-4a34-9170-49796875127b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:42.609128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24734602-0ed9-4f0b-b149-16cd634e10c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:42.610237 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24734602-0ed9-4f0b-b149-16cd634e10c4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:42.610723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24734602-0ed9-4f0b-b149-16cd634e10c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:42.611209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24734602-0ed9-4f0b-b149-16cd634e10c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:42.613095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24734602-0ed9-4f0b-b149-16cd634e10c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:42.614487 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24734602-0ed9-4f0b-b149-16cd634e10c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:42.616514 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24734602-0ed9-4f0b-b149-16cd634e10c4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:42.637726 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 312/635] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:42.638812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24734602-0ed9-4f0b-b149-16cd634e10c4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:42.733177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:42.747066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff025be-3bb0-4a34-9170-49796875127b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:42.774416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:42.809585 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fed26-1c21-48d6-a41d-14abc0d200ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:42.809585 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fed26-1c21-48d6-a41d-14abc0d200ca None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:42.809585 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fed26-1c21-48d6-a41d-14abc0d200ca None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:42.809585 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fed26-1c21-48d6-a41d-14abc0d200ca None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:42.811387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fed26-1c21-48d6-a41d-14abc0d200ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:42.812212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fed26-1c21-48d6-a41d-14abc0d200ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:42.815294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fed26-1c21-48d6-a41d-14abc0d200ca None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:42.837457 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 324/636] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:42.847296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fed26-1c21-48d6-a41d-14abc0d200ca None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:42.851623 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24734602-0ed9-4f0b-b149-16cd634e10c4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:42.924590 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c761464-60fd-42de-9740-bb2d8f1a7987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:42.925293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c761464-60fd-42de-9740-bb2d8f1a7987 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:42.925800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c761464-60fd-42de-9740-bb2d8f1a7987 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:42.926497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c761464-60fd-42de-9740-bb2d8f1a7987 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:42.927994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c761464-60fd-42de-9740-bb2d8f1a7987 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:42.928522 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c761464-60fd-42de-9740-bb2d8f1a7987 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:42.930886 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c761464-60fd-42de-9740-bb2d8f1a7987 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:42.971978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 313/637] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Aug 3 10:30:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:42.973510 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c761464-60fd-42de-9740-bb2d8f1a7987 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:42.985193 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fed26-1c21-48d6-a41d-14abc0d200ca None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:43.043439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67559c56-e545-46c6-a0e3-bea022c9a33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:43.044818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67559c56-e545-46c6-a0e3-bea022c9a33d None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:43.045371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67559c56-e545-46c6-a0e3-bea022c9a33d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:43.046351 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67559c56-e545-46c6-a0e3-bea022c9a33d None admin] PATH_INFO: `/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/a81ea5afe74b43879abc94b3cd134fd2/roles/1df33ad923a44a59b50dcacb6a75fefb` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:43.068808 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67559c56-e545-46c6-a0e3-bea022c9a33d None admin] RBAC: Authorizing `identity:create_grant(project_id=4272b859979d49c8b8f0c17ec9eb81bc, user_id=a81ea5afe74b43879abc94b3cd134fd2, role_id=1df33ad923a44a59b50dcacb6a75fefb)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:43.071307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67559c56-e545-46c6-a0e3-bea022c9a33d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:43.088313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:43.105567 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 325/638] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:30:42 2022] PUT /identity/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/a81ea5afe74b43879abc94b3cd134fd2/roles/1df33ad923a44a59b50dcacb6a75fefb => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:30:43.106635 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67559c56-e545-46c6-a0e3-bea022c9a33d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:43.195415 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c761464-60fd-42de-9740-bb2d8f1a7987 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:43.261286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:43.281973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6a677-a08c-49b3-ba5e-9a972908fba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:43.283539 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6a677-a08c-49b3-ba5e-9a972908fba7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:43.285419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6a677-a08c-49b3-ba5e-9a972908fba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:43.285419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6a677-a08c-49b3-ba5e-9a972908fba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:43.309543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f6a677-a08c-49b3-ba5e-9a972908fba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:43.311870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f6a677-a08c-49b3-ba5e-9a972908fba7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:43.341300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-a4f6a677-a08c-49b3-ba5e-9a972908fba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:43.341300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:43.341300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:43.341300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:43.341300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:43.341300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:43.341300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:43.341300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:43.341300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:43.341300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:43.341300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:43.341300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:43.341300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:43.341300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:43.341300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:43.342945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:43.345193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:43.345193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:43.345193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:43.345193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:43.345193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 314/639] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:43.349113 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6a677-a08c-49b3-ba5e-9a972908fba7 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:43.358598 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67559c56-e545-46c6-a0e3-bea022c9a33d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:43.425290 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bc7a2-ab27-421a-b7b5-360858cb0f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:43.426434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bc7a2-ab27-421a-b7b5-360858cb0f70 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:43.426849 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bc7a2-ab27-421a-b7b5-360858cb0f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:43.427130 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bc7a2-ab27-421a-b7b5-360858cb0f70 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:43.437520 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bc7a2-ab27-421a-b7b5-360858cb0f70 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:43.440549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bc7a2-ab27-421a-b7b5-360858cb0f70 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ad9bc7a2-ab27-421a-b7b5-360858cb0f70 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:43.461765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:43.464465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:43.467492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:43.467492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:43.467492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:43.467492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:43.467492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:43.467492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:43.467492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:43.467492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:43.467492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 03 10:30:43.467492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:43.467492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 326/640] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:43.467492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bc7a2-ab27-421a-b7b5-360858cb0f70 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:43.618157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:43.645164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6a677-a08c-49b3-ba5e-9a972908fba7 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:43.701880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bc7a2-ab27-421a-b7b5-360858cb0f70 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:43.732611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283abfe-6434-49eb-a962-754fa7bd982e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:43.734023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283abfe-6434-49eb-a962-754fa7bd982e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:43.734680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283abfe-6434-49eb-a962-754fa7bd982e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:43.735363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283abfe-6434-49eb-a962-754fa7bd982e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:43.749442 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a863d-18b0-4be0-aad0-aff278a373ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:43.750391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a863d-18b0-4be0-aad0-aff278a373ea None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:43.750817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a863d-18b0-4be0-aad0-aff278a373ea None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:43.751164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a863d-18b0-4be0-aad0-aff278a373ea None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:43.752019 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a863d-18b0-4be0-aad0-aff278a373ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:43.752957 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a863d-18b0-4be0-aad0-aff278a373ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:43.754610 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a863d-18b0-4be0-aad0-aff278a373ea None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:43.774776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 327/641] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:43.776093 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a863d-18b0-4be0-aad0-aff278a373ea None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:43.816193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:43.844647 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283abfe-6434-49eb-a962-754fa7bd982e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:43.846785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283abfe-6434-49eb-a962-754fa7bd982e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:43.990509 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a863d-18b0-4be0-aad0-aff278a373ea None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:44.025832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 315/642] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:44.028080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a955b4-c076-4acb-8a16-a17120f77d6a None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:44.028561 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a955b4-c076-4acb-8a16-a17120f77d6a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:44.029019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a955b4-c076-4acb-8a16-a17120f77d6a None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:44.031144 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:44.035941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 316/643] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:44.035941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a955b4-c076-4acb-8a16-a17120f77d6a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:44.064974 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af892265-43b7-4987-9b75-54c7b75e54b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:44.066610 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af892265-43b7-4987-9b75-54c7b75e54b7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:44.067088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af892265-43b7-4987-9b75-54c7b75e54b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:44.067538 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af892265-43b7-4987-9b75-54c7b75e54b7 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:44.068897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af892265-43b7-4987-9b75-54c7b75e54b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:44.069416 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af892265-43b7-4987-9b75-54c7b75e54b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:44.071532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af892265-43b7-4987-9b75-54c7b75e54b7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:44.098504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 328/644] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Aug 3 10:30:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:44.099988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af892265-43b7-4987-9b75-54c7b75e54b7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:44.182900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:44.236760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a955b4-c076-4acb-8a16-a17120f77d6a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:44.281905 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af892265-43b7-4987-9b75-54c7b75e54b7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:44.305338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0732b44-09b4-4544-8589-f8c649d8c262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:44.306556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0732b44-09b4-4544-8589-f8c649d8c262 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:44.307075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0732b44-09b4-4544-8589-f8c649d8c262 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:44.307515 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0732b44-09b4-4544-8589-f8c649d8c262 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:44.316560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0732b44-09b4-4544-8589-f8c649d8c262 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:44.318084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0732b44-09b4-4544-8589-f8c649d8c262 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-e0732b44-09b4-4544-8589-f8c649d8c262 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:44.332338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:44.333344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:44.334844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:44.334844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:44.334844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:44.334844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:44.334844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:44.334844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:44.334844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:44.334844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:44.334844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:30:44.334844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:44.338497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 317/645] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 3 10:30:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:44.340133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab22b2-dcb3-4a31-98ed-edcfbf0e3973 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:44.340420 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab22b2-dcb3-4a31-98ed-edcfbf0e3973 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:44.340695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab22b2-dcb3-4a31-98ed-edcfbf0e3973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:44.343057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8b919-f716-4567-8949-4e90a1d82cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:44.345625 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8b919-f716-4567-8949-4e90a1d82cac None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:44.346083 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8b919-f716-4567-8949-4e90a1d82cac None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:44.347078 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8b919-f716-4567-8949-4e90a1d82cac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:44.361415 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e4ab22b2-dcb3-4a31-98ed-edcfbf0e3973 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:44.363089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e4ab22b2-dcb3-4a31-98ed-edcfbf0e3973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:44.366330 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8b919-f716-4567-8949-4e90a1d82cac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:44.369134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8b919-f716-4567-8949-4e90a1d82cac None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:44.398525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-f1c8b919-f716-4567-8949-4e90a1d82cac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:44.398525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:44.398525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:44.398525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:44.398525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:44.398525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:44.398525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:44.398525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:44.398525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:44.398525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:44.398525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:44.398525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:44.398525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:44.398525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:44.398525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:44.399514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:44.400398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:44.400398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:44.400398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:44.400398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:44.402709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 329/646] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:44.402709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8b919-f716-4567-8949-4e90a1d82cac None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:44.456799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:44.461892 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 318/647] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:44.462913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab22b2-dcb3-4a31-98ed-edcfbf0e3973 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:44.564281 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:44.608569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8b919-f716-4567-8949-4e90a1d82cac None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:44.621803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab22b2-dcb3-4a31-98ed-edcfbf0e3973 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:44.664670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0dfa5-8240-43c4-a8e4-45e044678cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:44.667426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0dfa5-8240-43c4-a8e4-45e044678cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:44.668338 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0dfa5-8240-43c4-a8e4-45e044678cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:44.669099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0dfa5-8240-43c4-a8e4-45e044678cf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:44.670710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33903d8-10ac-4a41-8c3b-4b5ee8377fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:44.671408 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33903d8-10ac-4a41-8c3b-4b5ee8377fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:44.671682 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33903d8-10ac-4a41-8c3b-4b5ee8377fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:44.671956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33903d8-10ac-4a41-8c3b-4b5ee8377fa2 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:44.672766 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33903d8-10ac-4a41-8c3b-4b5ee8377fa2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:44.673071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33903d8-10ac-4a41-8c3b-4b5ee8377fa2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:44.674694 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33903d8-10ac-4a41-8c3b-4b5ee8377fa2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:44.696916 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 319/648] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 3 10:30:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:44.697812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33903d8-10ac-4a41-8c3b-4b5ee8377fa2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:44.840558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0dfa5-8240-43c4-a8e4-45e044678cf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:44.843249 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0dfa5-8240-43c4-a8e4-45e044678cf1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:44.847293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33903d8-10ac-4a41-8c3b-4b5ee8377fa2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:44.887491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:44.901908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:44.913037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1364a7c-64a1-497d-a344-cd8a1aa3fbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:44.914181 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1364a7c-64a1-497d-a344-cd8a1aa3fbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:44.914375 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1364a7c-64a1-497d-a344-cd8a1aa3fbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:44.914731 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1364a7c-64a1-497d-a344-cd8a1aa3fbb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:44.915522 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1364a7c-64a1-497d-a344-cd8a1aa3fbb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:44.915906 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1364a7c-64a1-497d-a344-cd8a1aa3fbb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:44.917589 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1364a7c-64a1-497d-a344-cd8a1aa3fbb3 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:44.931149 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 320/649] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:44.932097 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1364a7c-64a1-497d-a344-cd8a1aa3fbb3 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:45.031440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 330/650] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:45.033165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0dfa5-8240-43c4-a8e4-45e044678cf1 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:45.091023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1364a7c-64a1-497d-a344-cd8a1aa3fbb3 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:45.157900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795c2c5-1796-494e-9bed-b8289911660f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:45.159464 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795c2c5-1796-494e-9bed-b8289911660f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:45.160303 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795c2c5-1796-494e-9bed-b8289911660f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:45.160837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795c2c5-1796-494e-9bed-b8289911660f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:45.185682 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795c2c5-1796-494e-9bed-b8289911660f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:45.195203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0dfa5-8240-43c4-a8e4-45e044678cf1 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-6795c2c5-1796-494e-9bed-b8289911660f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:45.199517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:45.200498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:45.201353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:45.201353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:45.201353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:45.201353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:45.201353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:45.201353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:45.201353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:45.201353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:45.201353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:45.201353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:45.201353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:45.201353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:45.202229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 321/651] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:45.203752 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795c2c5-1796-494e-9bed-b8289911660f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:45.239281 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:45.248174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9205d-899c-4836-a6fd-574dd08da72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:45.249146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9205d-899c-4836-a6fd-574dd08da72b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:45.249603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9205d-899c-4836-a6fd-574dd08da72b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:45.250302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9205d-899c-4836-a6fd-574dd08da72b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:45.271584 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:45.337205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795c2c5-1796-494e-9bed-b8289911660f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:45.346812 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9205d-899c-4836-a6fd-574dd08da72b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:45.348857 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9205d-899c-4836-a6fd-574dd08da72b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:45.378995 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf20bc-1142-444c-87e0-56ba3f5de085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:45.379831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf20bc-1142-444c-87e0-56ba3f5de085 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:45.380119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf20bc-1142-444c-87e0-56ba3f5de085 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:45.380402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf20bc-1142-444c-87e0-56ba3f5de085 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:45.394833 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf20bc-1142-444c-87e0-56ba3f5de085 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:45.397324 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf20bc-1142-444c-87e0-56ba3f5de085 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:45.414687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-86cf20bc-1142-444c-87e0-56ba3f5de085 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:45.414687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:45.414687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:45.414687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:45.414687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:45.414687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:45.414687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:45.414687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:45.414687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:45.414687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:45.414687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:45.414687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:45.414687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:45.414687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:45.414687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:45.415734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:45.416658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:45.416658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:45.416658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:45.416658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:45.416658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:45.416658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:45.416658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:45.416658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:45.421904 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 322/652] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:45.422841 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf20bc-1142-444c-87e0-56ba3f5de085 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:45.504721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 331/653] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:45.505825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9205d-899c-4836-a6fd-574dd08da72b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:45.551491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:45.579727 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf20bc-1142-444c-87e0-56ba3f5de085 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:45.592085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:45.641551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78ceb4-2ddd-423d-943a-2808a98ac73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:45.643011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78ceb4-2ddd-423d-943a-2808a98ac73f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:45.643993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78ceb4-2ddd-423d-943a-2808a98ac73f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:45.644486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78ceb4-2ddd-423d-943a-2808a98ac73f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:45.658119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9205d-899c-4836-a6fd-574dd08da72b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:45.660579 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78ceb4-2ddd-423d-943a-2808a98ac73f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:45.663562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78ceb4-2ddd-423d-943a-2808a98ac73f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:45.702011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-fe78ceb4-2ddd-423d-943a-2808a98ac73f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:45.702011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:45.702011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:45.702011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:45.702011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:45.702011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:45.702011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:45.702011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:45.702011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:45.702011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:45.702011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:45.702011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:45.702011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:45.702011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:45.702011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:45.702955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:45.704004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:45.704004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:45.704004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:45.704004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:45.704004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 323/654] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:45.704556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78ceb4-2ddd-423d-943a-2808a98ac73f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:45.723047 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2541e2-6797-4854-ab73-0cc5defc8b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:45.724155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2541e2-6797-4854-ab73-0cc5defc8b78 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:45.724742 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2541e2-6797-4854-ab73-0cc5defc8b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:45.725340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2541e2-6797-4854-ab73-0cc5defc8b78 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:45.727154 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2541e2-6797-4854-ab73-0cc5defc8b78 None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:45.727723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2541e2-6797-4854-ab73-0cc5defc8b78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:45.730313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2541e2-6797-4854-ab73-0cc5defc8b78 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:45.747715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 332/655] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:45.749262 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2541e2-6797-4854-ab73-0cc5defc8b78 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:45.862540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78ceb4-2ddd-423d-943a-2808a98ac73f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:45.873604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:45.886759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2541e2-6797-4854-ab73-0cc5defc8b78 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:45.912870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb278c-a4f3-45e1-96e0-a0eb088450c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:45.913892 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb278c-a4f3-45e1-96e0-a0eb088450c2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:45.914342 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb278c-a4f3-45e1-96e0-a0eb088450c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:45.914880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb278c-a4f3-45e1-96e0-a0eb088450c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:45.916196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb278c-a4f3-45e1-96e0-a0eb088450c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:45.916788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb278c-a4f3-45e1-96e0-a0eb088450c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:45.919609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb278c-a4f3-45e1-96e0-a0eb088450c2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:45.939087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8d963-94dc-4189-908c-79c4218082ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:45.940674 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8d963-94dc-4189-908c-79c4218082ba None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:45.941145 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8d963-94dc-4189-908c-79c4218082ba None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:45.941638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8d963-94dc-4189-908c-79c4218082ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:45.943797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 324/656] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:45.945956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e9518-947a-41ca-94d0-ae454c50d192 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:45.946392 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e9518-947a-41ca-94d0-ae454c50d192 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:45.946809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e9518-947a-41ca-94d0-ae454c50d192 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:45.949204 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 325/657] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:45.954971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e9518-947a-41ca-94d0-ae454c50d192 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:45.959593 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8d963-94dc-4189-908c-79c4218082ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:45.961228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8d963-94dc-4189-908c-79c4218082ba None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:45.978586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-63b8d963-94dc-4189-908c-79c4218082ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:45.978586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:45.978586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:45.978586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:45.978586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:45.978586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:45.978586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:45.978586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:45.978586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:45.978586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:45.978586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:45.978586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:45.978586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:45.978586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:45.978586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:45.979579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:45.980852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:45.980852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:45.980852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:45.980852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:45.980852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 333/658] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:45.981155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8d963-94dc-4189-908c-79c4218082ba None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:45.983981 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:46.109012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e9518-947a-41ca-94d0-ae454c50d192 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:46.119623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8d963-94dc-4189-908c-79c4218082ba None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:46.149210 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3c7f3-0c8e-477b-bd3d-661b2bcdd3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:46.149886 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3c7f3-0c8e-477b-bd3d-661b2bcdd3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:46.150169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3c7f3-0c8e-477b-bd3d-661b2bcdd3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:46.150446 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3c7f3-0c8e-477b-bd3d-661b2bcdd3e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:46.151157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f3c7f3-0c8e-477b-bd3d-661b2bcdd3e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:46.151466 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f3c7f3-0c8e-477b-bd3d-661b2bcdd3e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:46.152864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f3c7f3-0c8e-477b-bd3d-661b2bcdd3e4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:46.166410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 326/659] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:46.167221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3c7f3-0c8e-477b-bd3d-661b2bcdd3e4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:46.178544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e47848-d72e-446d-ab3d-2246c9055a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:46.179488 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e47848-d72e-446d-ab3d-2246c9055a14 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:46.179774 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e47848-d72e-446d-ab3d-2246c9055a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:46.180054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e47848-d72e-446d-ab3d-2246c9055a14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:46.192594 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e47848-d72e-446d-ab3d-2246c9055a14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:46.194588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e47848-d72e-446d-ab3d-2246c9055a14 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:46.196849 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:46.212088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-03e47848-d72e-446d-ab3d-2246c9055a14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:46.212088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:46.212088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:46.212088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:46.212088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:46.212088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:46.212088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:46.212088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:46.212088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:46.212088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:46.212088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:46.212088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:46.212088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:46.212088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:46.212088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:46.213054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:46.214268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:46.214268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:46.214268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:46.214268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:46.214268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 334/660] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:46.219104 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e3cdb-3547-4066-85d8-c54658b190dd None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:46.219471 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e3cdb-3547-4066-85d8-c54658b190dd None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:46.219754 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e3cdb-3547-4066-85d8-c54658b190dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:46.242141 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ef5e3cdb-3547-4066-85d8-c54658b190dd None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:46.243405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ef5e3cdb-3547-4066-85d8-c54658b190dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:46.288014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:46.326736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3c7f3-0c8e-477b-bd3d-661b2bcdd3e4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:46.334795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 335/661] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:46.339168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e3cdb-3547-4066-85d8-c54658b190dd None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:46.385914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613490a-214e-40f4-b47c-da97a96b288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:46.387104 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613490a-214e-40f4-b47c-da97a96b288a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:46.387576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613490a-214e-40f4-b47c-da97a96b288a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:46.388043 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613490a-214e-40f4-b47c-da97a96b288a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:46.389452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613490a-214e-40f4-b47c-da97a96b288a None admin] RBAC: Adding query filter params (role.id=1df33ad923a44a59b50dcacb6a75fefb, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=3b5f547a388c4f64aeb95a7435818741) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:46.389963 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613490a-214e-40f4-b47c-da97a96b288a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:46.392181 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613490a-214e-40f4-b47c-da97a96b288a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:46.405664 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 327/662] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:30:46 2022] GET /identity/v3/role_assignments?user.id=3b5f547a388c4f64aeb95a7435818741&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=1df33ad923a44a59b50dcacb6a75fefb => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:46.406901 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613490a-214e-40f4-b47c-da97a96b288a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:46.523630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e3cdb-3547-4066-85d8-c54658b190dd None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:46.545314 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613490a-214e-40f4-b47c-da97a96b288a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:46.576054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:46.594057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f07a5e-faea-4301-b1fc-8903d194d539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:46.594710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f07a5e-faea-4301-b1fc-8903d194d539 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:46.594995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f07a5e-faea-4301-b1fc-8903d194d539 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:46.595270 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f07a5e-faea-4301-b1fc-8903d194d539 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:46.596657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f07a5e-faea-4301-b1fc-8903d194d539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:46.596976 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f07a5e-faea-4301-b1fc-8903d194d539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:46.598586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f07a5e-faea-4301-b1fc-8903d194d539 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:46.610832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d9971-fcf8-4671-812f-7d762924a587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:46.612741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d9971-fcf8-4671-812f-7d762924a587 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:46.613430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d9971-fcf8-4671-812f-7d762924a587 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:46.620899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d9971-fcf8-4671-812f-7d762924a587 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:46.621155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 336/663] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:46.621155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f07a5e-faea-4301-b1fc-8903d194d539 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:46.638232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d9971-fcf8-4671-812f-7d762924a587 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:46.642018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d9971-fcf8-4671-812f-7d762924a587 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:46.666842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-bc1d9971-fcf8-4671-812f-7d762924a587 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:46.666842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:46.666842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:46.666842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:46.666842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:46.666842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:46.666842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:46.666842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:46.666842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:46.666842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:46.666842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:46.666842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:46.666842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:46.666842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:46.666842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:46.668711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:46.670603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:46.670603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:46.670603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:46.670603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:46.670603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:46.670603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:46.670603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:46.670603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:46.670603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 328/664] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:46.676045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d9971-fcf8-4671-812f-7d762924a587 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:46.678784 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:46.782302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f07a5e-faea-4301-b1fc-8903d194d539 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:46.822373 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf1cf2-c68f-4d35-88f1-d3c0bc6b3cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:46.823036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf1cf2-c68f-4d35-88f1-d3c0bc6b3cae None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:46.823323 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf1cf2-c68f-4d35-88f1-d3c0bc6b3cae None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:46.823472 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d9971-fcf8-4671-812f-7d762924a587 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:46.823607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf1cf2-c68f-4d35-88f1-d3c0bc6b3cae None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:46.824284 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf1cf2-c68f-4d35-88f1-d3c0bc6b3cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:46.824674 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf1cf2-c68f-4d35-88f1-d3c0bc6b3cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:46.826146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf1cf2-c68f-4d35-88f1-d3c0bc6b3cae None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:46.857502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 337/665] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:46.857502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf1cf2-c68f-4d35-88f1-d3c0bc6b3cae None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:46.870162 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b1106-7a7a-48a3-99b5-7047fd70d43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:46.870978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b1106-7a7a-48a3-99b5-7047fd70d43d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:46.871301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b1106-7a7a-48a3-99b5-7047fd70d43d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:46.871549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b1106-7a7a-48a3-99b5-7047fd70d43d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:46.892365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b1106-7a7a-48a3-99b5-7047fd70d43d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-671b1106-7a7a-48a3-99b5-7047fd70d43d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:46.904133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:46.905125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:46.906054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:46.906054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:46.906054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:46.906054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:46.906054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:46.906054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:46.906054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:46.906054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:46.906054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:46.906054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:46.906054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:46.906054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:46.906707 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 329/666] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:46.908351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b1106-7a7a-48a3-99b5-7047fd70d43d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:46.934501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:47.022673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf1cf2-c68f-4d35-88f1-d3c0bc6b3cae None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:47.033773 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:47.076262 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d635245-ae7a-45dd-a51d-c5d01fdbbee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:47.077084 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d635245-ae7a-45dd-a51d-c5d01fdbbee3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:47.077704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d635245-ae7a-45dd-a51d-c5d01fdbbee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:47.077867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d635245-ae7a-45dd-a51d-c5d01fdbbee3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:47.087409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d635245-ae7a-45dd-a51d-c5d01fdbbee3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:47.089316 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d635245-ae7a-45dd-a51d-c5d01fdbbee3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:47.113604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-3d635245-ae7a-45dd-a51d-c5d01fdbbee3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:47.113604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:47.113604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:47.113604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:47.113604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:47.113604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:47.113604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:47.113604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:47.113604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:47.113604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:47.113604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:47.113604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:47.113604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:47.113604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:47.113604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:47.114820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:47.115710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:47.115710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:47.115710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:47.115710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:47.115710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:47.115710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:47.115710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:47.115710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:47.115710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 338/667] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:47.116698 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d635245-ae7a-45dd-a51d-c5d01fdbbee3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:47.117878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b1106-7a7a-48a3-99b5-7047fd70d43d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:47.189166 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddefc2-651d-4dfe-bc33-21db99f98d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:47.191318 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddefc2-651d-4dfe-bc33-21db99f98d49 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:47.192400 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddefc2-651d-4dfe-bc33-21db99f98d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:47.193190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddefc2-651d-4dfe-bc33-21db99f98d49 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:47.195187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afddefc2-651d-4dfe-bc33-21db99f98d49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:47.196023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afddefc2-651d-4dfe-bc33-21db99f98d49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:47.199209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afddefc2-651d-4dfe-bc33-21db99f98d49 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:47.225728 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 330/668] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:47.227919 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddefc2-651d-4dfe-bc33-21db99f98d49 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:47.316883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d635245-ae7a-45dd-a51d-c5d01fdbbee3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:47.323960 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:47.395580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53351bb-0321-460d-8e9b-f6f02c9d5055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:47.396519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53351bb-0321-460d-8e9b-f6f02c9d5055 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:47.396813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53351bb-0321-460d-8e9b-f6f02c9d5055 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:47.397092 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53351bb-0321-460d-8e9b-f6f02c9d5055 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:47.399012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddefc2-651d-4dfe-bc33-21db99f98d49 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:47.408629 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53351bb-0321-460d-8e9b-f6f02c9d5055 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:47.411358 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53351bb-0321-460d-8e9b-f6f02c9d5055 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:47.432427 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-e53351bb-0321-460d-8e9b-f6f02c9d5055 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:47.436444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:47.437412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:47.438381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:47.438381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:47.438381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:47.438381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:47.438381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:47.438381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:47.438381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:47.438381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:47.438381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 10:30:47.438381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:47.445598 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 339/669] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:47.445598 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53351bb-0321-460d-8e9b-f6f02c9d5055 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:47.453581 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0029ff8-6417-4d81-8937-bf43b74ec5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:47.454414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0029ff8-6417-4d81-8937-bf43b74ec5ff None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:47.454859 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0029ff8-6417-4d81-8937-bf43b74ec5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:47.455221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0029ff8-6417-4d81-8937-bf43b74ec5ff None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:47.456520 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0029ff8-6417-4d81-8937-bf43b74ec5ff None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:47.456944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0029ff8-6417-4d81-8937-bf43b74ec5ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:47.458619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0029ff8-6417-4d81-8937-bf43b74ec5ff None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:47.475647 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 331/670] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:47.476975 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0029ff8-6417-4d81-8937-bf43b74ec5ff None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:47.596402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53351bb-0321-460d-8e9b-f6f02c9d5055 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:47.633496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0029ff8-6417-4d81-8937-bf43b74ec5ff None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:47.655990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2d0b2-5b68-43ab-8852-07aff2483ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:47.657042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2d0b2-5b68-43ab-8852-07aff2483ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:47.657633 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2d0b2-5b68-43ab-8852-07aff2483ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:47.658142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2d0b2-5b68-43ab-8852-07aff2483ba7 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:47.660664 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2d0b2-5b68-43ab-8852-07aff2483ba7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:47.661219 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2d0b2-5b68-43ab-8852-07aff2483ba7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:47.665005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2d0b2-5b68-43ab-8852-07aff2483ba7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:47.681977 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed5f32-0875-4aed-a828-339aa8b1214d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:47.685171 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed5f32-0875-4aed-a828-339aa8b1214d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:47.685748 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed5f32-0875-4aed-a828-339aa8b1214d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:47.686239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed5f32-0875-4aed-a828-339aa8b1214d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:47.687287 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed5f32-0875-4aed-a828-339aa8b1214d None admin] RBAC: Adding query filter params (role.id=1df33ad923a44a59b50dcacb6a75fefb, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=300fb5e1c5c546808b935871673e3d0f) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:47.687808 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed5f32-0875-4aed-a828-339aa8b1214d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:47.689711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed5f32-0875-4aed-a828-339aa8b1214d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:47.700341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 340/671] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:47.702100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2d0b2-5b68-43ab-8852-07aff2483ba7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:47.705757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 332/672] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:30:47 2022] GET /identity/v3/role_assignments?user.id=300fb5e1c5c546808b935871673e3d0f&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=1df33ad923a44a59b50dcacb6a75fefb => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:47.707393 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed5f32-0875-4aed-a828-339aa8b1214d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:47.759843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:47.794015 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:47.889576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed5f32-0875-4aed-a828-339aa8b1214d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:47.965427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2d0b2-5b68-43ab-8852-07aff2483ba7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:47.969957 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30976460-7c72-4612-a974-10a6937de64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:47.970439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30976460-7c72-4612-a974-10a6937de64c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:47.972265 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30976460-7c72-4612-a974-10a6937de64c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:47.972265 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30976460-7c72-4612-a974-10a6937de64c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:47.991451 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30976460-7c72-4612-a974-10a6937de64c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:47.994106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30976460-7c72-4612-a974-10a6937de64c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:48.021077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-30976460-7c72-4612-a974-10a6937de64c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:48.021077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:48.021077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:48.021077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:48.021077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:48.021077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:48.021077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:48.021077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:48.021077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:48.021077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:48.021077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:48.021077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:48.021077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:48.021077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:48.021077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:48.022221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:48.023132 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:48.023132 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:48.023132 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:48.023132 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:48.023132 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 333/673] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:48.024377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30976460-7c72-4612-a974-10a6937de64c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:48.038964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc47e6-ef15-4531-b548-59ac0430781c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:48.043094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc47e6-ef15-4531-b548-59ac0430781c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:48.043094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc47e6-ef15-4531-b548-59ac0430781c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:48.043094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc47e6-ef15-4531-b548-59ac0430781c None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:48.045175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc47e6-ef15-4531-b548-59ac0430781c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:48.046087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc47e6-ef15-4531-b548-59ac0430781c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:48.050073 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc47e6-ef15-4531-b548-59ac0430781c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:48.077173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 341/674] 10.222.0.47 () {56 vars in 1139 bytes} [Wed Aug 3 10:30:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:48.078286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc47e6-ef15-4531-b548-59ac0430781c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:48.257147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:48.258051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:48.271192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30976460-7c72-4612-a974-10a6937de64c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:48.292590 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc47e6-ef15-4531-b548-59ac0430781c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:48.341269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace077f1-fbb6-4095-8657-e7dd4e6be477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:48.344612 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b97b9-1dd5-4b8a-89ad-d91d6220f737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:48.344612 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b97b9-1dd5-4b8a-89ad-d91d6220f737 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:48.344612 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b97b9-1dd5-4b8a-89ad-d91d6220f737 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:48.344612 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b97b9-1dd5-4b8a-89ad-d91d6220f737 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:48.345032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace077f1-fbb6-4095-8657-e7dd4e6be477 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:48.345032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace077f1-fbb6-4095-8657-e7dd4e6be477 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:48.345032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace077f1-fbb6-4095-8657-e7dd4e6be477 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:48.346537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace077f1-fbb6-4095-8657-e7dd4e6be477 None admin] RBAC: Adding query filter params (role.id=1df33ad923a44a59b50dcacb6a75fefb, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=e66f25cc329f418eba6bd637bf3e8a3a) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:48.347212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b97b9-1dd5-4b8a-89ad-d91d6220f737 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:48.347526 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b97b9-1dd5-4b8a-89ad-d91d6220f737 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:48.348123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace077f1-fbb6-4095-8657-e7dd4e6be477 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:48.351507 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace077f1-fbb6-4095-8657-e7dd4e6be477 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:48.352746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b97b9-1dd5-4b8a-89ad-d91d6220f737 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:48.385503 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 334/675] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:30:48 2022] GET /identity/v3/role_assignments?user.id=e66f25cc329f418eba6bd637bf3e8a3a&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=1df33ad923a44a59b50dcacb6a75fefb => generated 608 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:48.385503 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace077f1-fbb6-4095-8657-e7dd4e6be477 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:48.393397 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 342/676] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:48.398530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b97b9-1dd5-4b8a-89ad-d91d6220f737 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:48.591992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b97b9-1dd5-4b8a-89ad-d91d6220f737 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:48.643949 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d3cd3-36d8-4be7-8e0b-bdb8b3f3c07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:48.648679 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d3cd3-36d8-4be7-8e0b-bdb8b3f3c07d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:48.648679 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d3cd3-36d8-4be7-8e0b-bdb8b3f3c07d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:48.648679 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d3cd3-36d8-4be7-8e0b-bdb8b3f3c07d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:48.657880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d3cd3-36d8-4be7-8e0b-bdb8b3f3c07d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:48.660839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d3cd3-36d8-4be7-8e0b-bdb8b3f3c07d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-1a7d3cd3-36d8-4be7-8e0b-bdb8b3f3c07d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:48.679067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:48.680880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:48.682750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:48.682750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:48.682750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:48.682750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:48.682750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:48.682750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:48.682750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:48.682750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:48.682750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 10:30:48.682750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:48.692234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 343/677] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:30:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:48.692234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d3cd3-36d8-4be7-8e0b-bdb8b3f3c07d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:48.692234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:48.835499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace077f1-fbb6-4095-8657-e7dd4e6be477 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:48.922367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d3cd3-36d8-4be7-8e0b-bdb8b3f3c07d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:48.938574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99ae69-d92c-4c9c-bb9d-2a3ed57986d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:48.939847 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99ae69-d92c-4c9c-bb9d-2a3ed57986d7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:48.940699 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99ae69-d92c-4c9c-bb9d-2a3ed57986d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:48.941012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99ae69-d92c-4c9c-bb9d-2a3ed57986d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:48.965765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ed15c-8b72-46df-9024-d7602ba29d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:48.967557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ed15c-8b72-46df-9024-d7602ba29d53 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:48.968054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ed15c-8b72-46df-9024-d7602ba29d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:48.968854 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ed15c-8b72-46df-9024-d7602ba29d53 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:48.970270 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ed15c-8b72-46df-9024-d7602ba29d53 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:48.970809 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ed15c-8b72-46df-9024-d7602ba29d53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:48.973251 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ed15c-8b72-46df-9024-d7602ba29d53 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:49.003703 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 344/678] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Aug 3 10:30:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:49.003703 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35304e8d-4196-4af0-854d-9f34a1fed0de None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:49.003703 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35304e8d-4196-4af0-854d-9f34a1fed0de None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:49.003703 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35304e8d-4196-4af0-854d-9f34a1fed0de None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:49.005178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 345/679] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:49.006370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35304e8d-4196-4af0-854d-9f34a1fed0de None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:49.019669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:49.099918 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99ae69-d92c-4c9c-bb9d-2a3ed57986d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:49.103837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99ae69-d92c-4c9c-bb9d-2a3ed57986d7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:49.125002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:49.263826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35304e8d-4196-4af0-854d-9f34a1fed0de None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:49.324427 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 335/680] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 943 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:49.326700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aef535-015f-475c-8fa1-ba8ac3b3b46e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:49.326700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aef535-015f-475c-8fa1-ba8ac3b3b46e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:49.326700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aef535-015f-475c-8fa1-ba8ac3b3b46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:49.340508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69756ba4-20c2-4606-94cc-1785c8c246d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:49.344250 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69756ba4-20c2-4606-94cc-1785c8c246d5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:49.344250 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69756ba4-20c2-4606-94cc-1785c8c246d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:49.344479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69756ba4-20c2-4606-94cc-1785c8c246d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:49.385316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a8aef535-015f-475c-8fa1-ba8ac3b3b46e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:49.388967 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a8aef535-015f-475c-8fa1-ba8ac3b3b46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:49.509499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69756ba4-20c2-4606-94cc-1785c8c246d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:49.509499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69756ba4-20c2-4606-94cc-1785c8c246d5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:49.603950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:49.630093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 336/681] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:49.638815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aef535-015f-475c-8fa1-ba8ac3b3b46e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:49.666785 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:49.782636 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 346/682] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 776 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:49.782860 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69756ba4-20c2-4606-94cc-1785c8c246d5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:49.955106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aef535-015f-475c-8fa1-ba8ac3b3b46e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:49.975139 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69756ba4-20c2-4606-94cc-1785c8c246d5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:50.029591 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8617151-f86b-475d-97e9-39280bff6e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:50.030313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8617151-f86b-475d-97e9-39280bff6e62 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:50.030781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8617151-f86b-475d-97e9-39280bff6e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:50.031238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8617151-f86b-475d-97e9-39280bff6e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:50.044879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8617151-f86b-475d-97e9-39280bff6e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:50.046536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8617151-f86b-475d-97e9-39280bff6e62 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:50.063596 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7ccbb-d5d5-4701-9a37-3bf5b0c111b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:50.065362 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7ccbb-d5d5-4701-9a37-3bf5b0c111b9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:50.065944 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7ccbb-d5d5-4701-9a37-3bf5b0c111b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:50.066441 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7ccbb-d5d5-4701-9a37-3bf5b0c111b9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:50.067351 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7ccbb-d5d5-4701-9a37-3bf5b0c111b9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:50.069295 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7ccbb-d5d5-4701-9a37-3bf5b0c111b9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:50.086058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-a8617151-f86b-475d-97e9-39280bff6e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:50.086058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:50.086058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:50.086058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:50.086058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:50.086058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:50.086058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:50.086058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:50.086058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:50.086058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:50.086058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:50.086058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:50.086058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:50.086058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:50.086058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:50.087119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:50.088182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:50.088182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:50.088182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:50.088182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:50.090962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 337/683] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 459 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:50.090962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8617151-f86b-475d-97e9-39280bff6e62 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-1ee7ccbb-d5d5-4701-9a37-3bf5b0c111b9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:50.114319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:50.115532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:50.116561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:50.116561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:50.116561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:50.116561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:50.116561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:50.116561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:50.116561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:50.116561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:50.116561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:50.116561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:50.116561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:50.116561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:30:50.116561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:30:50.116561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 10:30:50.116561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:50.119221 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 347/684] 10.222.0.47 () {56 vars in 1129 bytes} [Wed Aug 3 10:30:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:50.123722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7ccbb-d5d5-4701-9a37-3bf5b0c111b9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:50.190627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:50.192416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:50.264277 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8617151-f86b-475d-97e9-39280bff6e62 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:50.330427 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e39039-ead2-43a9-a378-9e8c004c8817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:50.331540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e39039-ead2-43a9-a378-9e8c004c8817 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:50.331991 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e39039-ead2-43a9-a378-9e8c004c8817 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:50.332483 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e39039-ead2-43a9-a378-9e8c004c8817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:50.341151 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7ccbb-d5d5-4701-9a37-3bf5b0c111b9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:50.368672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e39039-ead2-43a9-a378-9e8c004c8817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:50.371484 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e39039-ead2-43a9-a378-9e8c004c8817 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:50.390912 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3ad74-03c8-4759-8017-a70c4766e841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:50.391907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3ad74-03c8-4759-8017-a70c4766e841 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:50.392410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3ad74-03c8-4759-8017-a70c4766e841 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:50.392926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3ad74-03c8-4759-8017-a70c4766e841 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:50.393896 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3ad74-03c8-4759-8017-a70c4766e841 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:50.394434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3ad74-03c8-4759-8017-a70c4766e841 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:50.396191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3ad74-03c8-4759-8017-a70c4766e841 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:50.398844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-f7e39039-ead2-43a9-a378-9e8c004c8817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:50.398844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:50.398844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:50.398844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:50.398844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:50.398844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:50.398844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:50.398844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:50.398844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:50.398844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:50.398844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:50.398844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:50.398844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:50.398844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:50.398844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:50.399757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:50.400638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:50.400638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:50.400638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:50.400638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:50.405758 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 338/685] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:50.405758 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e39039-ead2-43a9-a378-9e8c004c8817 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:50.427704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 348/686] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:50.436273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95054f-8f56-4960-a3c2-36636d844d42 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:50.436707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95054f-8f56-4960-a3c2-36636d844d42 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:50.437134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95054f-8f56-4960-a3c2-36636d844d42 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:50.439551 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 349/687] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:50.441381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95054f-8f56-4960-a3c2-36636d844d42 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:50.588529 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:50.598796 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e39039-ead2-43a9-a378-9e8c004c8817 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:50.616112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:50.652873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95054f-8f56-4960-a3c2-36636d844d42 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:50.656207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2fe41-6399-4451-9b20-c5fef4600985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:50.656888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2fe41-6399-4451-9b20-c5fef4600985 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:50.658286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2fe41-6399-4451-9b20-c5fef4600985 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:50.659300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2fe41-6399-4451-9b20-c5fef4600985 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:50.660053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2fe41-6399-4451-9b20-c5fef4600985 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:50.660369 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2fe41-6399-4451-9b20-c5fef4600985 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:50.661804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2fe41-6399-4451-9b20-c5fef4600985 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:50.666871 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 339/688] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Aug 3 10:30:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:50.667709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2fe41-6399-4451-9b20-c5fef4600985 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:50.737733 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab577f-1174-4617-9f42-281fff0baa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:50.738975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab577f-1174-4617-9f42-281fff0baa19 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:50.739564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab577f-1174-4617-9f42-281fff0baa19 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:50.740206 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab577f-1174-4617-9f42-281fff0baa19 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:50.741697 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab577f-1174-4617-9f42-281fff0baa19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:50.742387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab577f-1174-4617-9f42-281fff0baa19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:50.745102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab577f-1174-4617-9f42-281fff0baa19 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:50.777201 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 350/689] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:50.779681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afa784-419d-41e1-994c-2bb41c6cd70e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:50.780416 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afa784-419d-41e1-994c-2bb41c6cd70e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:50.780909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afa784-419d-41e1-994c-2bb41c6cd70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:50.814392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d6afa784-419d-41e1-994c-2bb41c6cd70e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:50.816127 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d6afa784-419d-41e1-994c-2bb41c6cd70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:50.854698 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2fe41-6399-4451-9b20-c5fef4600985 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:50.913270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b565b-ef7c-473c-a77d-be24f762a2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:50.914499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b565b-ef7c-473c-a77d-be24f762a2df None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:50.915069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b565b-ef7c-473c-a77d-be24f762a2df None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:50.915637 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b565b-ef7c-473c-a77d-be24f762a2df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:50.928616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b565b-ef7c-473c-a77d-be24f762a2df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:50.931034 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b565b-ef7c-473c-a77d-be24f762a2df None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:50.942038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 351/690] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:50.944024 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afa784-419d-41e1-994c-2bb41c6cd70e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:50.950920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-117b565b-ef7c-473c-a77d-be24f762a2df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:50.950920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:50.950920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:50.950920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:50.950920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:50.950920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:50.950920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:50.950920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:50.950920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:50.950920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:50.950920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:50.950920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:50.950920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:50.950920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:50.950920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:50.962084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:50.964337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:50.964337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:50.964337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:50.964337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:50.964337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:50.964337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:50.964337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:50.964337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:50.964337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 340/691] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:50.964337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b565b-ef7c-473c-a77d-be24f762a2df None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:51.010596 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:51.106179 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:51.134090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afa784-419d-41e1-994c-2bb41c6cd70e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:51.167155 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b565b-ef7c-473c-a77d-be24f762a2df None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:51.190114 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467acc4-8c30-4e90-9d66-ee1974fc0ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:51.191047 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467acc4-8c30-4e90-9d66-ee1974fc0ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:51.191611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467acc4-8c30-4e90-9d66-ee1974fc0ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:51.191794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467acc4-8c30-4e90-9d66-ee1974fc0ea0 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:51.195856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467acc4-8c30-4e90-9d66-ee1974fc0ea0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:51.198290 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467acc4-8c30-4e90-9d66-ee1974fc0ea0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:51.202329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467acc4-8c30-4e90-9d66-ee1974fc0ea0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:51.210303 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca7281-a744-470c-8443-7989a9f46c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:51.210966 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca7281-a744-470c-8443-7989a9f46c02 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:51.211252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca7281-a744-470c-8443-7989a9f46c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:51.211532 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca7281-a744-470c-8443-7989a9f46c02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:51.212703 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 352/692] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Aug 3 10:30:50 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:51.218622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467acc4-8c30-4e90-9d66-ee1974fc0ea0 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:51.219854 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca7281-a744-470c-8443-7989a9f46c02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:51.222085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca7281-a744-470c-8443-7989a9f46c02 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:51.235115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-f7ca7281-a744-470c-8443-7989a9f46c02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:51.235115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:51.235115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:51.235115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:51.235115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:51.235115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:51.235115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:51.235115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:51.235115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:51.235115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:51.235115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:51.235115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:51.235115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:51.235115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:51.235115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:51.236799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:51.238431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:51.238431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:51.238431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:51.238431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:51.238431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:51.238431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:51.238431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:51.238431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:51.241665 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 341/693] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:51.242682 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca7281-a744-470c-8443-7989a9f46c02 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:51.366198 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:51.390277 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca7281-a744-470c-8443-7989a9f46c02 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:51.404552 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467acc4-8c30-4e90-9d66-ee1974fc0ea0 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:51.441154 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d043bc5-d337-46c5-89d2-b158e3facb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:51.441914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d043bc5-d337-46c5-89d2-b158e3facb5d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:51.442183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d043bc5-d337-46c5-89d2-b158e3facb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:51.442456 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d043bc5-d337-46c5-89d2-b158e3facb5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:51.449500 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf61da-8b75-4432-bf56-81f604b2cbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:51.450424 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf61da-8b75-4432-bf56-81f604b2cbcf None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:51.450907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf61da-8b75-4432-bf56-81f604b2cbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:51.451388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf61da-8b75-4432-bf56-81f604b2cbcf None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:51.452341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf61da-8b75-4432-bf56-81f604b2cbcf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:51.452846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf61da-8b75-4432-bf56-81f604b2cbcf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:51.453856 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d043bc5-d337-46c5-89d2-b158e3facb5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:51.454686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf61da-8b75-4432-bf56-81f604b2cbcf None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:51.455865 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d043bc5-d337-46c5-89d2-b158e3facb5d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:51.464639 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:51.470323 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 353/694] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:51.471665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88561108-c1d9-48d4-be7a-46f4044e4f65 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:51.472402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88561108-c1d9-48d4-be7a-46f4044e4f65 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:51.472659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-1d043bc5-d337-46c5-89d2-b158e3facb5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:51.472659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:51.472659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:51.472659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:51.472659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:51.472659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:51.472659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:51.472659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:51.472659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:51.472659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:51.472659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:51.472659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:51.472659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:51.472659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:51.472659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:51.473541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:51.478984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:51.478984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:51.478984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:51.478984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:51.479216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88561108-c1d9-48d4-be7a-46f4044e4f65 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:51.479216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 354/695] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:51.479216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88561108-c1d9-48d4-be7a-46f4044e4f65 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:51.479411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 342/696] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:51.479891 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d043bc5-d337-46c5-89d2-b158e3facb5d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:51.615930 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d043bc5-d337-46c5-89d2-b158e3facb5d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:51.617624 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88561108-c1d9-48d4-be7a-46f4044e4f65 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:51.664646 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30843a-5d46-44d5-a581-1a9726247aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:51.665274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30843a-5d46-44d5-a581-1a9726247aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:51.665586 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30843a-5d46-44d5-a581-1a9726247aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:51.665872 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30843a-5d46-44d5-a581-1a9726247aa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:51.666648 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30843a-5d46-44d5-a581-1a9726247aa1 None admin] RBAC: Adding query filter params (role.id=1df33ad923a44a59b50dcacb6a75fefb, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=1c211cc19a2045bd9383d31c7e81ceac) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:51.666955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30843a-5d46-44d5-a581-1a9726247aa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:51.668401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30843a-5d46-44d5-a581-1a9726247aa1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:51.669087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e417e-4600-49ad-a533-e75039b6eabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:51.669743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e417e-4600-49ad-a533-e75039b6eabe None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:51.670106 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e417e-4600-49ad-a533-e75039b6eabe None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:51.670446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e417e-4600-49ad-a533-e75039b6eabe None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:51.671257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e417e-4600-49ad-a533-e75039b6eabe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:51.671749 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e417e-4600-49ad-a533-e75039b6eabe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:51.672351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:51.673194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e417e-4600-49ad-a533-e75039b6eabe None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:51.685965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 343/697] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:30:51 2022] GET /identity/v3/role_assignments?user.id=1c211cc19a2045bd9383d31c7e81ceac&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=1df33ad923a44a59b50dcacb6a75fefb => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:51.688011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81c7ea-2e33-420a-a944-3884f5ced2f2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:51.689264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81c7ea-2e33-420a-a944-3884f5ced2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:51.689264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81c7ea-2e33-420a-a944-3884f5ced2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:51.693529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 355/698] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:51.698978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e417e-4600-49ad-a533-e75039b6eabe None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:51.726951 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8f81c7ea-2e33-420a-a944-3884f5ced2f2 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:51.728496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8f81c7ea-2e33-420a-a944-3884f5ced2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:51.808634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:51.866588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 344/699] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:51.867267 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e417e-4600-49ad-a533-e75039b6eabe None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:51.867855 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81c7ea-2e33-420a-a944-3884f5ced2f2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:51.912609 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfb8ed-d67e-422f-a071-d173f9e01548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:51.912609 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfb8ed-d67e-422f-a071-d173f9e01548 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:51.912609 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfb8ed-d67e-422f-a071-d173f9e01548 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:51.912609 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfb8ed-d67e-422f-a071-d173f9e01548 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:51.913179 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cfb8ed-d67e-422f-a071-d173f9e01548 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:51.913537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cfb8ed-d67e-422f-a071-d173f9e01548 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:51.914977 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cfb8ed-d67e-422f-a071-d173f9e01548 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:51.928925 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 356/700] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:51.929812 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfb8ed-d67e-422f-a071-d173f9e01548 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:52.014169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81c7ea-2e33-420a-a944-3884f5ced2f2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:52.043818 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:52.053977 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa4507-7c30-4e66-a803-94391b22f368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:52.054609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa4507-7c30-4e66-a803-94391b22f368 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:52.054889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa4507-7c30-4e66-a803-94391b22f368 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:52.055164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa4507-7c30-4e66-a803-94391b22f368 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:52.055971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa4507-7c30-4e66-a803-94391b22f368 None admin] RBAC: Adding query filter params (role.id=1df33ad923a44a59b50dcacb6a75fefb, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=a81ea5afe74b43879abc94b3cd134fd2) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:52.056278 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa4507-7c30-4e66-a803-94391b22f368 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:52.057967 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa4507-7c30-4e66-a803-94391b22f368 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:52.061198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfb8ed-d67e-422f-a071-d173f9e01548 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:52.070594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 345/701] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:30:51 2022] GET /identity/v3/role_assignments?user.id=a81ea5afe74b43879abc94b3cd134fd2&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=1df33ad923a44a59b50dcacb6a75fefb => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:52.071800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa4507-7c30-4e66-a803-94391b22f368 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:52.083533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:52.104445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec457380-5e59-46f9-9d4e-e43fce00df13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:52.105608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec457380-5e59-46f9-9d4e-e43fce00df13 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:52.106120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec457380-5e59-46f9-9d4e-e43fce00df13 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:52.106598 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec457380-5e59-46f9-9d4e-e43fce00df13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:52.121398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec457380-5e59-46f9-9d4e-e43fce00df13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ec457380-5e59-46f9-9d4e-e43fce00df13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:52.126899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:52.127934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:52.128769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:52.128769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:52.128769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:52.128769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:52.128769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:52.128769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:52.128769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:52.128769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:52.128769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:52.128769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:52.128769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:30:52.128769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:52.131886 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 357/702] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:30:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:52.142950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec457380-5e59-46f9-9d4e-e43fce00df13 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:52.264932 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa4507-7c30-4e66-a803-94391b22f368 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:52.312186 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec457380-5e59-46f9-9d4e-e43fce00df13 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:52.381045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec0b42-903b-4085-b420-72a3cf9166c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:52.382552 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec0b42-903b-4085-b420-72a3cf9166c7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:52.383331 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec0b42-903b-4085-b420-72a3cf9166c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:52.384114 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec0b42-903b-4085-b420-72a3cf9166c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:52.385828 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec0b42-903b-4085-b420-72a3cf9166c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:52.387448 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec0b42-903b-4085-b420-72a3cf9166c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:52.391773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec0b42-903b-4085-b420-72a3cf9166c7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:52.398500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086aae5-6604-4c9c-8fea-87b8f8c9e899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:52.399139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086aae5-6604-4c9c-8fea-87b8f8c9e899 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:52.399416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086aae5-6604-4c9c-8fea-87b8f8c9e899 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:52.399675 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086aae5-6604-4c9c-8fea-87b8f8c9e899 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:52.400372 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086aae5-6604-4c9c-8fea-87b8f8c9e899 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:52.403084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086aae5-6604-4c9c-8fea-87b8f8c9e899 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:52.412094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 358/703] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:52.420022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec0b42-903b-4085-b420-72a3cf9166c7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:52.454880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 346/704] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:30:52 2022] POST /identity/v3/users => generated 254 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:30:52.515672 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:52.580024 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec0b42-903b-4085-b420-72a3cf9166c7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:52.628827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff173d-4597-4df7-81cd-986f375cfb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:52.629539 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff173d-4597-4df7-81cd-986f375cfb33 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:52.629827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff173d-4597-4df7-81cd-986f375cfb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:52.630115 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff173d-4597-4df7-81cd-986f375cfb33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:52.645386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff173d-4597-4df7-81cd-986f375cfb33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:52.646800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff173d-4597-4df7-81cd-986f375cfb33 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:52.665371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-e7ff173d-4597-4df7-81cd-986f375cfb33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:52.665371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:52.665371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:52.665371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:52.665371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:52.665371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:52.665371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:52.665371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:52.665371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:52.665371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:52.665371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:52.665371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:52.665371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:52.665371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:52.665371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:52.666887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:52.668473 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:52.668473 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:52.668473 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:52.668473 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:52.668473 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 359/705] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:52.679477 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086aae5-6604-4c9c-8fea-87b8f8c9e899 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:52.810154 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:52.912325 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086aae5-6604-4c9c-8fea-87b8f8c9e899 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:52.952660 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02a86a-d38e-4816-9412-1dc2926c27ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:52.953299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02a86a-d38e-4816-9412-1dc2926c27ff None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:52.953614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02a86a-d38e-4816-9412-1dc2926c27ff None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:52.953890 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02a86a-d38e-4816-9412-1dc2926c27ff None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:52.954608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02a86a-d38e-4816-9412-1dc2926c27ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:52.954914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02a86a-d38e-4816-9412-1dc2926c27ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:52.956894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02a86a-d38e-4816-9412-1dc2926c27ff None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:52.974769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 347/706] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:52.976241 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff173d-4597-4df7-81cd-986f375cfb33 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:53.143222 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:53.160740 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff173d-4597-4df7-81cd-986f375cfb33 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:53.210802 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c7a42-6d1d-4fa4-95ed-8515545a325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:53.211867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c7a42-6d1d-4fa4-95ed-8515545a325c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:53.212391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c7a42-6d1d-4fa4-95ed-8515545a325c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:53.212879 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c7a42-6d1d-4fa4-95ed-8515545a325c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:53.226506 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c7a42-6d1d-4fa4-95ed-8515545a325c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:53.228343 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c7a42-6d1d-4fa4-95ed-8515545a325c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-618c7a42-6d1d-4fa4-95ed-8515545a325c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:53.247298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:53.248370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:53.249394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:53.249394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:53.249394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:53.249394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:53.249394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:53.249394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:53.249394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:53.249394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:53.249394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:30:53.249394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:53.251931 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 360/707] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 3 10:30:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:53.257902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02a86a-d38e-4816-9412-1dc2926c27ff None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:53.368134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02a86a-d38e-4816-9412-1dc2926c27ff None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:53.378776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:53.407480 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157203c8-ecf8-40c1-b901-4f8298ec7455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:53.408128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157203c8-ecf8-40c1-b901-4f8298ec7455 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:53.409001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157203c8-ecf8-40c1-b901-4f8298ec7455 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:53.409244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157203c8-ecf8-40c1-b901-4f8298ec7455 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:53.410099 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157203c8-ecf8-40c1-b901-4f8298ec7455 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:53.410407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157203c8-ecf8-40c1-b901-4f8298ec7455 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:53.411881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157203c8-ecf8-40c1-b901-4f8298ec7455 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:53.432602 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 348/708] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 3 10:30:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:53.442632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c7a42-6d1d-4fa4-95ed-8515545a325c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:53.577207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c7a42-6d1d-4fa4-95ed-8515545a325c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:53.633720 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97053f07-a2e9-4806-a03f-1b5409b78164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:53.634661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97053f07-a2e9-4806-a03f-1b5409b78164 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:53.634960 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97053f07-a2e9-4806-a03f-1b5409b78164 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:53.635243 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97053f07-a2e9-4806-a03f-1b5409b78164 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:53.674268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:53.759617 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97053f07-a2e9-4806-a03f-1b5409b78164 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:53.761445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97053f07-a2e9-4806-a03f-1b5409b78164 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:53.957941 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 361/709] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:53.966476 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157203c8-ecf8-40c1-b901-4f8298ec7455 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:54.107502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cc31e-f9f3-49c6-a66d-1ea14db052a8 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:54.107963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cc31e-f9f3-49c6-a66d-1ea14db052a8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:54.108429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cc31e-f9f3-49c6-a66d-1ea14db052a8 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:54.111156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 362/710] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:54.146480 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157203c8-ecf8-40c1-b901-4f8298ec7455 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:54.151232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa0b7e-4f15-46a9-a809-2630321b3086 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:54.151736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa0b7e-4f15-46a9-a809-2630321b3086 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:54.151825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa0b7e-4f15-46a9-a809-2630321b3086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:54.198646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-86aa0b7e-4f15-46a9-a809-2630321b3086 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:54.199875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-86aa0b7e-4f15-46a9-a809-2630321b3086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:54.209707 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07292502-7339-4ff4-8836-f99115ba0ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:54.210719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07292502-7339-4ff4-8836-f99115ba0ade None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:54.211207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07292502-7339-4ff4-8836-f99115ba0ade None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:54.211696 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07292502-7339-4ff4-8836-f99115ba0ade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:54.231353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07292502-7339-4ff4-8836-f99115ba0ade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:54.234149 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07292502-7339-4ff4-8836-f99115ba0ade None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:54.260080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-07292502-7339-4ff4-8836-f99115ba0ade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:54.260080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:54.260080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:54.260080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:54.260080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:54.260080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:54.260080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:54.260080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:54.260080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:54.260080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:54.260080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:54.260080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:54.260080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:54.260080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:54.260080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:54.261448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:54.262351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:54.262351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:54.262351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:54.262351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:54.263016 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 349/711] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:54.272934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07292502-7339-4ff4-8836-f99115ba0ade None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:54.313882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:54.330438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:54.350453 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 363/712] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:54.419874 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa0b7e-4f15-46a9-a809-2630321b3086 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:54.463104 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07292502-7339-4ff4-8836-f99115ba0ade None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:54.539961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27f4cf-84fd-42ae-b2da-773e3a3c4b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:54.541054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27f4cf-84fd-42ae-b2da-773e3a3c4b1e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:54.541577 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27f4cf-84fd-42ae-b2da-773e3a3c4b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:54.542782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27f4cf-84fd-42ae-b2da-773e3a3c4b1e None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:54.544253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27f4cf-84fd-42ae-b2da-773e3a3c4b1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:54.544990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27f4cf-84fd-42ae-b2da-773e3a3c4b1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:54.548146 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27f4cf-84fd-42ae-b2da-773e3a3c4b1e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:54.575319 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 350/713] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:30:54.583297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27f4cf-84fd-42ae-b2da-773e3a3c4b1e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:54.709978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa0b7e-4f15-46a9-a809-2630321b3086 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:54.775690 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a7afb-12fe-4795-9b0d-fc67df3ff6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:54.776662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a7afb-12fe-4795-9b0d-fc67df3ff6fd None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:54.777159 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a7afb-12fe-4795-9b0d-fc67df3ff6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:54.777687 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a7afb-12fe-4795-9b0d-fc67df3ff6fd None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:54.779012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a7afb-12fe-4795-9b0d-fc67df3ff6fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:54.781758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a7afb-12fe-4795-9b0d-fc67df3ff6fd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:54.802169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:54.817695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 364/714] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 3 10:30:54 2022] POST /identity/v3/services => generated 251 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:30:54.826071 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f196-27fb-440b-b20f-ef4157d64c08 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:54.827318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f196-27fb-440b-b20f-ef4157d64c08 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:54.828100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f196-27fb-440b-b20f-ef4157d64c08 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:54.828975 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27f4cf-84fd-42ae-b2da-773e3a3c4b1e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:54.832775 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 365/715] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:54 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:54.840334 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc374ac-1483-4657-a79e-928da3c2422f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:54.841181 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc374ac-1483-4657-a79e-928da3c2422f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:54.841987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc374ac-1483-4657-a79e-928da3c2422f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:54.891047 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-bbc374ac-1483-4657-a79e-928da3c2422f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:54.892289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bbc374ac-1483-4657-a79e-928da3c2422f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:54.895983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452a7cd-4e60-4f86-bf3c-26379bc1db75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:54.896650 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452a7cd-4e60-4f86-bf3c-26379bc1db75 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:54.896941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452a7cd-4e60-4f86-bf3c-26379bc1db75 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:54.897215 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452a7cd-4e60-4f86-bf3c-26379bc1db75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:54.916994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452a7cd-4e60-4f86-bf3c-26379bc1db75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:54.919359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452a7cd-4e60-4f86-bf3c-26379bc1db75 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:54.950439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-5452a7cd-4e60-4f86-bf3c-26379bc1db75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:54.950439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:54.950439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:54.950439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:54.950439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:54.950439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:54.950439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:54.950439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:54.950439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:54.950439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:54.950439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:54.950439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:54.950439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:54.950439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:54.950439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:54.952462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:54.954786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:54.954786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:54.954786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:54.954786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:54.954786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:54.954786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:54.954786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:54.954786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:54.959352 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 351/716] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:54.965009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:54.969069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452a7cd-4e60-4f86-bf3c-26379bc1db75 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:55.055611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 366/717] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:54 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:55.093569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc374ac-1483-4657-a79e-928da3c2422f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:55.170094 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452a7cd-4e60-4f86-bf3c-26379bc1db75 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:55.232487 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629393d-d2e6-41f2-8d6b-d413695eae1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:55.233196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629393d-d2e6-41f2-8d6b-d413695eae1e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:55.233524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629393d-d2e6-41f2-8d6b-d413695eae1e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:55.233785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629393d-d2e6-41f2-8d6b-d413695eae1e None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:55.234554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629393d-d2e6-41f2-8d6b-d413695eae1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:55.234862 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629393d-d2e6-41f2-8d6b-d413695eae1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:55.236781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629393d-d2e6-41f2-8d6b-d413695eae1e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:55.238488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:55.254105 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 352/718] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:30:55.260215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc374ac-1483-4657-a79e-928da3c2422f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:55.261211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629393d-d2e6-41f2-8d6b-d413695eae1e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:55.307043 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad54c43-59c7-4e48-ba49-60326695d82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:55.307628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad54c43-59c7-4e48-ba49-60326695d82d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:55.307911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad54c43-59c7-4e48-ba49-60326695d82d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:55.308186 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad54c43-59c7-4e48-ba49-60326695d82d None admin] PATH_INFO: `/v3/services/network` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:55.308827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad54c43-59c7-4e48-ba49-60326695d82d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:55.310237 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad54c43-59c7-4e48-ba49-60326695d82d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-dad54c43-59c7-4e48-ba49-60326695d82d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:55.337716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:55.338762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:55.339847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:55.339847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:55.339847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:55.339847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:55.339847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:55.339847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:55.339847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:55.339847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:55.339847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:55.339847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:55.339847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:30:55.339847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:30:55.339847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 10:30:55.339847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:55.341303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 367/719] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:55 2022] GET /identity/v3/services/network => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:55.343039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fd242-d636-4c2a-8f90-44c241157d4e None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:55.343413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fd242-d636-4c2a-8f90-44c241157d4e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:55.343911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fd242-d636-4c2a-8f90-44c241157d4e None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:55.346569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 368/720] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:55.348041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fd242-d636-4c2a-8f90-44c241157d4e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:55.407721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:55.417710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629393d-d2e6-41f2-8d6b-d413695eae1e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:55.460093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e98ac-6377-40c2-b1df-c9353fa6402f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:55.461210 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e98ac-6377-40c2-b1df-c9353fa6402f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:55.461877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e98ac-6377-40c2-b1df-c9353fa6402f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:55.462396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e98ac-6377-40c2-b1df-c9353fa6402f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:55.463453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e98ac-6377-40c2-b1df-c9353fa6402f None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=e66f25cc329f418eba6bd637bf3e8a3a) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:55.464262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e98ac-6377-40c2-b1df-c9353fa6402f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:55.466432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e98ac-6377-40c2-b1df-c9353fa6402f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:55.475371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 353/721] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:30:55 2022] GET /identity/v3/role_assignments?user.id=e66f25cc329f418eba6bd637bf3e8a3a&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=01714797cab143e1a7a150c068fa9315 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:55.477151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ec57d-7061-4257-89de-0ee59288a7b3 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:55.477687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ec57d-7061-4257-89de-0ee59288a7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:55.478194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ec57d-7061-4257-89de-0ee59288a7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:55.490486 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fd242-d636-4c2a-8f90-44c241157d4e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:55.500425 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ed9ec57d-7061-4257-89de-0ee59288a7b3 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:55.502054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ed9ec57d-7061-4257-89de-0ee59288a7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:55.572440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bae43-20b9-4bcd-9131-a430a5d44459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:55.574081 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bae43-20b9-4bcd-9131-a430a5d44459 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:55.574709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bae43-20b9-4bcd-9131-a430a5d44459 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:55.576462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bae43-20b9-4bcd-9131-a430a5d44459 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:55.577941 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680bae43-20b9-4bcd-9131-a430a5d44459 None admin] RBAC: Adding query filter params (name=network) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:55.578811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680bae43-20b9-4bcd-9131-a430a5d44459 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:55.580977 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680bae43-20b9-4bcd-9131-a430a5d44459 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:55.594257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 369/722] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 3 10:30:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:55.603015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bae43-20b9-4bcd-9131-a430a5d44459 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:55.635927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:55.671377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 354/723] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:55.695768 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ec57d-7061-4257-89de-0ee59288a7b3 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:55.804104 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:55.810184 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bae43-20b9-4bcd-9131-a430a5d44459 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:55.852018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ec57d-7061-4257-89de-0ee59288a7b3 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:55.857802 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748af3c7-3451-422c-b08b-01d1db231bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:55.858441 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748af3c7-3451-422c-b08b-01d1db231bff None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:55.858720 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748af3c7-3451-422c-b08b-01d1db231bff None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:55.858993 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748af3c7-3451-422c-b08b-01d1db231bff None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:55.859682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748af3c7-3451-422c-b08b-01d1db231bff None admin] RBAC: Adding query filter params (type=network) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:55.859986 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748af3c7-3451-422c-b08b-01d1db231bff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:55.861382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748af3c7-3451-422c-b08b-01d1db231bff None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:55.868114 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 370/724] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 3 10:30:55 2022] GET /identity/v3/services?type=network => generated 124 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:55.869426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c6c36-1880-40aa-8149-fc9ed14509d5 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:55.869848 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c6c36-1880-40aa-8149-fc9ed14509d5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:55.870277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c6c36-1880-40aa-8149-fc9ed14509d5 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:55.875778 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 371/725] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:55.878560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02f67b-b245-421b-bdb0-43bb1ff9e91d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:55.878841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02f67b-b245-421b-bdb0-43bb1ff9e91d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:55.879116 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02f67b-b245-421b-bdb0-43bb1ff9e91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:55.902707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ed02f67b-b245-421b-bdb0-43bb1ff9e91d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:55.904368 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ed02f67b-b245-421b-bdb0-43bb1ff9e91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:56.001650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 372/726] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 10:30:56.002823 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83943a-b5c3-44de-b0d8-7aaabc0a63f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:56.003825 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83943a-b5c3-44de-b0d8-7aaabc0a63f5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:56.004223 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83943a-b5c3-44de-b0d8-7aaabc0a63f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:56.004721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83943a-b5c3-44de-b0d8-7aaabc0a63f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:56.018662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02f67b-b245-421b-bdb0-43bb1ff9e91d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:56.045231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83943a-b5c3-44de-b0d8-7aaabc0a63f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-8a83943a-b5c3-44de-b0d8-7aaabc0a63f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:56.051020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:56.052017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:56.052857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:56.052857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:56.052857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:56.052857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:56.052857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:56.052857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:56.052857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:56.052857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:56.052857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:56.052857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:56.052857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:30:56.052857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:56.058004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 355/727] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:30:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:30:56.066369 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83943a-b5c3-44de-b0d8-7aaabc0a63f5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:56.178776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:56.185071 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:56.202845 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02f67b-b245-421b-bdb0-43bb1ff9e91d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:56.262091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83943a-b5c3-44de-b0d8-7aaabc0a63f5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:56.276695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7434ed2-60e2-4997-9a8f-3460e9b20cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:56.279706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7434ed2-60e2-4997-9a8f-3460e9b20cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:56.280498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7434ed2-60e2-4997-9a8f-3460e9b20cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:56.281170 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7434ed2-60e2-4997-9a8f-3460e9b20cb7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:56.303816 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7434ed2-60e2-4997-9a8f-3460e9b20cb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-a7434ed2-60e2-4997-9a8f-3460e9b20cb7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:56.314627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:56.316359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:56.318165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:56.318165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:56.318165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:56.318165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:56.318165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:56.318165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:56.318165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:56.318165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:56.318165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:56.318165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:56.318165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:30:56.318165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:56.329443 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 373/728] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:30:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:56.334786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7434ed2-60e2-4997-9a8f-3460e9b20cb7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:56.406911 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca1728-f828-4b7d-8d46-8fdce3ebd2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:56.407562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca1728-f828-4b7d-8d46-8fdce3ebd2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:56.407849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca1728-f828-4b7d-8d46-8fdce3ebd2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:56.408130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca1728-f828-4b7d-8d46-8fdce3ebd2f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:56.409482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca1728-f828-4b7d-8d46-8fdce3ebd2f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:56.409804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca1728-f828-4b7d-8d46-8fdce3ebd2f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:56.411218 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca1728-f828-4b7d-8d46-8fdce3ebd2f0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:56.433135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 356/729] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:56.438648 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca1728-f828-4b7d-8d46-8fdce3ebd2f0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:56.500302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7434ed2-60e2-4997-9a8f-3460e9b20cb7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:56.555689 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fff6f-9201-46b1-a6ea-aa3022f10aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:56.556707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fff6f-9201-46b1-a6ea-aa3022f10aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:56.557146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fff6f-9201-46b1-a6ea-aa3022f10aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:56.557632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fff6f-9201-46b1-a6ea-aa3022f10aa2 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:56.558585 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91fff6f-9201-46b1-a6ea-aa3022f10aa2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:56.559286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91fff6f-9201-46b1-a6ea-aa3022f10aa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:56.561384 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91fff6f-9201-46b1-a6ea-aa3022f10aa2 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:56.581810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 374/730] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:30:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:30:56.584302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fff6f-9201-46b1-a6ea-aa3022f10aa2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:56.612406 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:56.725046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca1728-f828-4b7d-8d46-8fdce3ebd2f0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:56.751325 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:56.780912 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fff6f-9201-46b1-a6ea-aa3022f10aa2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:56.793109 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176ad5f-0c08-4d09-9a10-7b2a8d2dfd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:56.793819 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176ad5f-0c08-4d09-9a10-7b2a8d2dfd1e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:56.794106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176ad5f-0c08-4d09-9a10-7b2a8d2dfd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:56.794384 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176ad5f-0c08-4d09-9a10-7b2a8d2dfd1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:56.822256 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176ad5f-0c08-4d09-9a10-7b2a8d2dfd1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:56.824746 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176ad5f-0c08-4d09-9a10-7b2a8d2dfd1e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:56.831469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcee03-8776-4ff8-9f4a-b4387cd9c616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:56.832738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcee03-8776-4ff8-9f4a-b4387cd9c616 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:56.833229 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcee03-8776-4ff8-9f4a-b4387cd9c616 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:56.833928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcee03-8776-4ff8-9f4a-b4387cd9c616 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:56.856638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-e176ad5f-0c08-4d09-9a10-7b2a8d2dfd1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:56.856638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:56.856638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:56.856638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:56.856638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:56.856638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:56.856638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:56.856638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:56.856638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:56.856638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:56.856638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:56.856638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:56.856638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:56.856638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:56.856638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:56.857638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:56.858542 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:56.858542 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:56.858542 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:56.858542 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:56.864896 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcee03-8776-4ff8-9f4a-b4387cd9c616 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:56.866909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcee03-8776-4ff8-9f4a-b4387cd9c616 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:56.872360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 357/731] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:56.882547 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176ad5f-0c08-4d09-9a10-7b2a8d2dfd1e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:56.885906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-75bcee03-8776-4ff8-9f4a-b4387cd9c616 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:56.885906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:56.885906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:56.885906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:56.885906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:56.885906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:56.885906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:56.885906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:56.885906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:56.885906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:56.885906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:56.885906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:56.885906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:56.885906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:56.885906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:56.886850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:56.887756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:56.887756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:56.887756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:56.887756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:56.893678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 375/732] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:56.896919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcee03-8776-4ff8-9f4a-b4387cd9c616 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:56.970966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:57.012962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcee03-8776-4ff8-9f4a-b4387cd9c616 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:57.054181 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66244da6-6ef0-458e-beac-e72161186e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:57.055233 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66244da6-6ef0-458e-beac-e72161186e45 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:57.055739 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66244da6-6ef0-458e-beac-e72161186e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:57.056215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66244da6-6ef0-458e-beac-e72161186e45 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:57.057156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66244da6-6ef0-458e-beac-e72161186e45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:57.057711 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66244da6-6ef0-458e-beac-e72161186e45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:57.059326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66244da6-6ef0-458e-beac-e72161186e45 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:57.079816 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 376/733] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:30:57.081667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66244da6-6ef0-458e-beac-e72161186e45 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:57.165286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176ad5f-0c08-4d09-9a10-7b2a8d2dfd1e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:57.202322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66244da6-6ef0-458e-beac-e72161186e45 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:57.229892 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c101b5-afcd-419f-be7f-7887bb3b2d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:57.231068 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c101b5-afcd-419f-be7f-7887bb3b2d42 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:57.231725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c101b5-afcd-419f-be7f-7887bb3b2d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:57.232314 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c101b5-afcd-419f-be7f-7887bb3b2d42 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:57.233524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c101b5-afcd-419f-be7f-7887bb3b2d42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:57.234178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c101b5-afcd-419f-be7f-7887bb3b2d42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:57.237069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c101b5-afcd-419f-be7f-7887bb3b2d42 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:57.240650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:57.256389 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca841028-98f3-4fc3-aa75-473d52303ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:57.257646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca841028-98f3-4fc3-aa75-473d52303ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:57.258260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca841028-98f3-4fc3-aa75-473d52303ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:57.258857 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca841028-98f3-4fc3-aa75-473d52303ed3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:57.270135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 358/734] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:57.273487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca841028-98f3-4fc3-aa75-473d52303ed3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:57.276300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c101b5-afcd-419f-be7f-7887bb3b2d42 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:57.276496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca841028-98f3-4fc3-aa75-473d52303ed3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ca841028-98f3-4fc3-aa75-473d52303ed3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:57.305574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:57.308042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:57.309983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:57.309983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:57.309983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:57.309983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:57.309983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:57.309983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:57.309983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:57.309983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:57.309983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 10:30:57.309983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:57.311198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 377/735] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 3 10:30:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:30:57.315338 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca841028-98f3-4fc3-aa75-473d52303ed3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:57.368540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:57.427924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca841028-98f3-4fc3-aa75-473d52303ed3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:57.467047 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd197428-f1f1-4ce8-969c-e5e5dcd0c85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:57.467883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd197428-f1f1-4ce8-969c-e5e5dcd0c85f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:57.468260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd197428-f1f1-4ce8-969c-e5e5dcd0c85f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:57.468697 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd197428-f1f1-4ce8-969c-e5e5dcd0c85f None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:57.477502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd197428-f1f1-4ce8-969c-e5e5dcd0c85f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:57.477502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd197428-f1f1-4ce8-969c-e5e5dcd0c85f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:57.477502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd197428-f1f1-4ce8-969c-e5e5dcd0c85f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:57.498334 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 378/736] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Aug 3 10:30:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:57.501505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd197428-f1f1-4ce8-969c-e5e5dcd0c85f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:57.572560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c101b5-afcd-419f-be7f-7887bb3b2d42 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:57.586285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:57.648414 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd197428-f1f1-4ce8-969c-e5e5dcd0c85f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:57.653398 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d56393-01e3-4f22-9de3-63a572a4b04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:57.654864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d56393-01e3-4f22-9de3-63a572a4b04e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:57.655597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d56393-01e3-4f22-9de3-63a572a4b04e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:57.656355 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d56393-01e3-4f22-9de3-63a572a4b04e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:57.671427 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d56393-01e3-4f22-9de3-63a572a4b04e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:57.675365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d56393-01e3-4f22-9de3-63a572a4b04e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:57.693864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d6cd0-1fdd-4336-8504-2a8f4dd3c546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:57.694516 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d6cd0-1fdd-4336-8504-2a8f4dd3c546 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:57.694800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d6cd0-1fdd-4336-8504-2a8f4dd3c546 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:57.695075 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d6cd0-1fdd-4336-8504-2a8f4dd3c546 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-39d56393-01e3-4f22-9de3-63a572a4b04e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:30:57.707473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:57.709316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:57.711193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:57.711193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:57.711193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:30:57.711193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:30:57.711193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:30:57.711193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:30:57.711193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:30:57.711193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:30:57.711193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 10:30:57.711193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:57.711193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 359/737] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:30:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:30:57.713180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0568607-be75-4ae6-9e6a-642b88dfd006 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:57.713180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0568607-be75-4ae6-9e6a-642b88dfd006 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:57.713180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0568607-be75-4ae6-9e6a-642b88dfd006 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:57.714936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 360/738] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:57.723402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d05717-9fff-4e75-ae48-7c9b6477bc7c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:57.723849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d05717-9fff-4e75-ae48-7c9b6477bc7c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:57.724101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d05717-9fff-4e75-ae48-7c9b6477bc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:57.748239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-81d05717-9fff-4e75-ae48-7c9b6477bc7c None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:57.750177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-81d05717-9fff-4e75-ae48-7c9b6477bc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:57.773669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d6cd0-1fdd-4336-8504-2a8f4dd3c546 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:57.775340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d6cd0-1fdd-4336-8504-2a8f4dd3c546 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:57.836047 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:57.852413 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 361/739] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:57.852413 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d05717-9fff-4e75-ae48-7c9b6477bc7c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:57.885120 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:57.963064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 379/740] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:57 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:57.965094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d6cd0-1fdd-4336-8504-2a8f4dd3c546 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:58.001004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d05717-9fff-4e75-ae48-7c9b6477bc7c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:58.056899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc464940-5ada-4f13-84f5-51a9d3486ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:58.057804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc464940-5ada-4f13-84f5-51a9d3486ecf None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:58.058263 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc464940-5ada-4f13-84f5-51a9d3486ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:58.058721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc464940-5ada-4f13-84f5-51a9d3486ecf None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:58.060032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc464940-5ada-4f13-84f5-51a9d3486ecf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:58.060645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc464940-5ada-4f13-84f5-51a9d3486ecf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:58.062741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc464940-5ada-4f13-84f5-51a9d3486ecf None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:58.097288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 362/741] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Aug 3 10:30:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:58.097288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3d566-638a-45c0-bd9d-c7d6c7f3e2ef None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:58.097288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3d566-638a-45c0-bd9d-c7d6c7f3e2ef None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:58.097288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3d566-638a-45c0-bd9d-c7d6c7f3e2ef None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:58.097288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 363/742] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:58.097288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3d566-638a-45c0-bd9d-c7d6c7f3e2ef None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:58.121534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d6cd0-1fdd-4336-8504-2a8f4dd3c546 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:58.175030 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0f59f-7c54-4c01-b69d-0a8fa8ade6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:58.176128 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0f59f-7c54-4c01-b69d-0a8fa8ade6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:58.176628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0f59f-7c54-4c01-b69d-0a8fa8ade6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:58.177120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0f59f-7c54-4c01-b69d-0a8fa8ade6f8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:58.178282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0f59f-7c54-4c01-b69d-0a8fa8ade6f8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:58.180849 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0f59f-7c54-4c01-b69d-0a8fa8ade6f8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:58.184509 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-aaa0f59f-7c54-4c01-b69d-0a8fa8ade6f8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:58.199225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:58.200425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:58.201425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 380/743] 10.222.0.47 () {56 vars in 1129 bytes} [Wed Aug 3 10:30:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:58.202896 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0f59f-7c54-4c01-b69d-0a8fa8ade6f8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:58.226902 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:58.234520 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3d566-638a-45c0-bd9d-c7d6c7f3e2ef None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:58.275689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3315fa7-d685-4032-9c72-63f543bef032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:58.276309 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3315fa7-d685-4032-9c72-63f543bef032 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:58.276566 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3315fa7-d685-4032-9c72-63f543bef032 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:58.276887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3315fa7-d685-4032-9c72-63f543bef032 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:58.288248 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3315fa7-d685-4032-9c72-63f543bef032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:58.289738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3315fa7-d685-4032-9c72-63f543bef032 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:58.386414 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0f59f-7c54-4c01-b69d-0a8fa8ade6f8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:58.426396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-d3315fa7-d685-4032-9c72-63f543bef032 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:58.426396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:58.426396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:58.426396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:58.426396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:58.426396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:58.426396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:58.426396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:58.426396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:58.426396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:58.426396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:58.426396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:58.426396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:58.426396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:58.426396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:58.428183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:58.430680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:58.430680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:58.430680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:58.430680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:58.430680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 364/744] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:58.430680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321841c-0afa-461e-b507-a6fe16303499 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:58.430680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321841c-0afa-461e-b507-a6fe16303499 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:58.430680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321841c-0afa-461e-b507-a6fe16303499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:58.449165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a321841c-0afa-461e-b507-a6fe16303499 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:58.450402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a321841c-0afa-461e-b507-a6fe16303499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:58.457636 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46e4b4-3fe8-4ce3-bba6-fd3263224aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:58.458595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46e4b4-3fe8-4ce3-bba6-fd3263224aeb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:58.459041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46e4b4-3fe8-4ce3-bba6-fd3263224aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:58.459478 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46e4b4-3fe8-4ce3-bba6-fd3263224aeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:58.553577 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 365/745] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:58 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:58.553577 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321841c-0afa-461e-b507-a6fe16303499 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:58.568446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46e4b4-3fe8-4ce3-bba6-fd3263224aeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:58.571297 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46e4b4-3fe8-4ce3-bba6-fd3263224aeb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:58.591073 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:58.641006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:58.718312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321841c-0afa-461e-b507-a6fe16303499 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:58.752083 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 381/746] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:30:58 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 10:30:58.754620 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46e4b4-3fe8-4ce3-bba6-fd3263224aeb None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:58.768603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b1c2e-5801-40e0-ae19-4851cf5a159f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:58.770051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b1c2e-5801-40e0-ae19-4851cf5a159f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:58.770422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b1c2e-5801-40e0-ae19-4851cf5a159f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:58.770775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b1c2e-5801-40e0-ae19-4851cf5a159f None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:58.771928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b1c2e-5801-40e0-ae19-4851cf5a159f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:58.772392 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b1c2e-5801-40e0-ae19-4851cf5a159f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:58.774516 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b1c2e-5801-40e0-ae19-4851cf5a159f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:58.789565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 366/747] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Aug 3 10:30:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:58.789565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b1c2e-5801-40e0-ae19-4851cf5a159f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:58.914161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46e4b4-3fe8-4ce3-bba6-fd3263224aeb None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:58.934579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:58.965884 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b1c2e-5801-40e0-ae19-4851cf5a159f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:58.966182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac179c-8506-4cbc-9d2e-36545579c092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:58.966532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac179c-8506-4cbc-9d2e-36545579c092 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:58.966812 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac179c-8506-4cbc-9d2e-36545579c092 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:58.967085 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac179c-8506-4cbc-9d2e-36545579c092 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:58.967783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac179c-8506-4cbc-9d2e-36545579c092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:58.968091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac179c-8506-4cbc-9d2e-36545579c092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:58.969502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac179c-8506-4cbc-9d2e-36545579c092 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:58.982442 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 382/748] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:58.983307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac179c-8506-4cbc-9d2e-36545579c092 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:58.989876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:59.010730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce5459-e8c6-43f2-aab8-48a0917c80de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:59.011388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce5459-e8c6-43f2-aab8-48a0917c80de None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:59.011671 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce5459-e8c6-43f2-aab8-48a0917c80de None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:59.011947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce5459-e8c6-43f2-aab8-48a0917c80de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:59.023061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce5459-e8c6-43f2-aab8-48a0917c80de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-f4ce5459-e8c6-43f2-aab8-48a0917c80de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:30:59.028053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:59.029153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:59.030076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:59.030076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:59.030076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:59.030076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:59.030076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:30:59.030076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:30:59.030076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:30:59.030076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:30:59.030076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:30:59.030076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:30:59.030076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:30:59.030076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:59.030076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 367/749] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:30:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:59.030076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce5459-e8c6-43f2-aab8-48a0917c80de None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:59.121479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac179c-8506-4cbc-9d2e-36545579c092 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:59.164577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738fc49-fb8f-4ac4-9366-edd55acb78a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:59.165150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738fc49-fb8f-4ac4-9366-edd55acb78a0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:59.165435 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738fc49-fb8f-4ac4-9366-edd55acb78a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:59.165741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738fc49-fb8f-4ac4-9366-edd55acb78a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:59.178192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738fc49-fb8f-4ac4-9366-edd55acb78a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:59.179653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738fc49-fb8f-4ac4-9366-edd55acb78a0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:59.184294 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce5459-e8c6-43f2-aab8-48a0917c80de None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:59.201499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-4738fc49-fb8f-4ac4-9366-edd55acb78a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:59.201499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:59.201499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:59.201499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:59.201499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:59.201499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:59.201499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:59.201499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:59.201499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:59.201499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:59.201499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:59.201499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:59.201499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:59.201499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:59.201499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:59.202543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:59.203418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:59.203418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:59.203418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:59.203418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:59.203418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 383/750] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:59.204357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f927c1-3769-4f93-b7bd-b65f4fd0c753 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:59.204726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f927c1-3769-4f93-b7bd-b65f4fd0c753 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:59.206127 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f927c1-3769-4f93-b7bd-b65f4fd0c753 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:59.209548 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 384/751] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:30:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:30:59.209800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f927c1-3769-4f93-b7bd-b65f4fd0c753 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:59.236273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a7c77-8eff-46f7-8264-3a134a19c739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:59.236900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a7c77-8eff-46f7-8264-3a134a19c739 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:59.237176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a7c77-8eff-46f7-8264-3a134a19c739 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:59.237452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a7c77-8eff-46f7-8264-3a134a19c739 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:59.238182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a7c77-8eff-46f7-8264-3a134a19c739 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:59.238486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a7c77-8eff-46f7-8264-3a134a19c739 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:59.239938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a7c77-8eff-46f7-8264-3a134a19c739 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:59.247729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 368/752] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Aug 3 10:30:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:59.248490 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:59.248628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a7c77-8eff-46f7-8264-3a134a19c739 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:59.303144 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:59.355669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f927c1-3769-4f93-b7bd-b65f4fd0c753 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:59.369346 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a7c77-8eff-46f7-8264-3a134a19c739 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:59.394926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b3d42-5df7-424d-9242-633f3451d9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:59.395573 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3d42-5df7-424d-9242-633f3451d9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:59.395858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3d42-5df7-424d-9242-633f3451d9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:59.396132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3d42-5df7-424d-9242-633f3451d9e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:59.403466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b3d42-5df7-424d-9242-633f3451d9e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:59.404924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b3d42-5df7-424d-9242-633f3451d9e6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:59.410687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7c318-626d-4a17-87e8-bed0c62f20dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:59.411742 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7c318-626d-4a17-87e8-bed0c62f20dd None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:59.411742 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7c318-626d-4a17-87e8-bed0c62f20dd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:59.411932 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7c318-626d-4a17-87e8-bed0c62f20dd None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:59.412555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7c318-626d-4a17-87e8-bed0c62f20dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:59.412856 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7c318-626d-4a17-87e8-bed0c62f20dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:59.414249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7c318-626d-4a17-87e8-bed0c62f20dd None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:59.417800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-3f0b3d42-5df7-424d-9242-633f3451d9e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:59.417800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:59.417800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:59.417800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:59.417800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:59.417800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:59.417800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:59.417800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:59.417800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:59.417800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:59.417800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:59.417800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:59.417800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:30:59.417800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:30:59.417800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:59.418873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:59.424736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:30:59.424736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:30:59.424736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:30:59.424736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:30:59.424736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:30:59.424736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:30:59.424736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:30:59.424736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:30:59.424736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 385/753] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:59.424736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b3d42-5df7-424d-9242-633f3451d9e6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:59.430798 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 369/754] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:30:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:59.432893 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0055a-1862-4532-a4dc-e00912bb980f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:59.432893 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0055a-1862-4532-a4dc-e00912bb980f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:59.433186 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0055a-1862-4532-a4dc-e00912bb980f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:59.452689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-47b0055a-1862-4532-a4dc-e00912bb980f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:30:59.453942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-47b0055a-1862-4532-a4dc-e00912bb980f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:30:59.515079 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:59.532079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 370/755] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:30:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 10:30:59.532940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0055a-1862-4532-a4dc-e00912bb980f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:59.537717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b3d42-5df7-424d-9242-633f3451d9e6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:59.574873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe5f3b-4ea1-4210-aea6-2b66bf4de7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:59.575490 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe5f3b-4ea1-4210-aea6-2b66bf4de7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:59.575771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe5f3b-4ea1-4210-aea6-2b66bf4de7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:59.576041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe5f3b-4ea1-4210-aea6-2b66bf4de7c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:59.576741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe5f3b-4ea1-4210-aea6-2b66bf4de7c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:59.577043 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe5f3b-4ea1-4210-aea6-2b66bf4de7c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:59.578453 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe5f3b-4ea1-4210-aea6-2b66bf4de7c7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:59.584743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:59.592139 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 386/756] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:59.593001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe5f3b-4ea1-4210-aea6-2b66bf4de7c7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:59.677171 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0055a-1862-4532-a4dc-e00912bb980f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:59.709248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe5f3b-4ea1-4210-aea6-2b66bf4de7c7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:59.717511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef710154-561d-4125-b195-32365a6e68fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:59.718150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef710154-561d-4125-b195-32365a6e68fb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:59.718443 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef710154-561d-4125-b195-32365a6e68fb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:59.718701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef710154-561d-4125-b195-32365a6e68fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:59.719464 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef710154-561d-4125-b195-32365a6e68fb None admin] RBAC: Adding query filter params (interface=public, service_id=10248adf4b2b40ce8b04c6185552c2e7, region_id=RegionOne) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:59.719774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef710154-561d-4125-b195-32365a6e68fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:59.721769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef710154-561d-4125-b195-32365a6e68fb None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:59.729792 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 371/757] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 3 10:30:59 2022] GET /identity/v3/endpoints?service_id=10248adf4b2b40ce8b04c6185552c2e7&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:59.730673 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef710154-561d-4125-b195-32365a6e68fb None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:59.754882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd5cbb-a353-4eb8-b9fc-5083ff624ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:59.755535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd5cbb-a353-4eb8-b9fc-5083ff624ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:59.755818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd5cbb-a353-4eb8-b9fc-5083ff624ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:59.756102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd5cbb-a353-4eb8-b9fc-5083ff624ec7 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:59.756924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd5cbb-a353-4eb8-b9fc-5083ff624ec7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:30:59.757232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd5cbb-a353-4eb8-b9fc-5083ff624ec7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:59.758769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd5cbb-a353-4eb8-b9fc-5083ff624ec7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:59.774626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 387/758] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:30:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:30:59.775688 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd5cbb-a353-4eb8-b9fc-5083ff624ec7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:59.779038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:59.855274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:30:59.863004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef710154-561d-4125-b195-32365a6e68fb None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:59.904021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd181e-016d-4d9e-80d5-2ff88794dcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:59.904285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd5cbb-a353-4eb8-b9fc-5083ff624ec7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:30:59.904692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd181e-016d-4d9e-80d5-2ff88794dcdf None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:59.905070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd181e-016d-4d9e-80d5-2ff88794dcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:59.905238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd181e-016d-4d9e-80d5-2ff88794dcdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:59.917107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd181e-016d-4d9e-80d5-2ff88794dcdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:59.918614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd181e-016d-4d9e-80d5-2ff88794dcdf None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:59.934572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-fefd181e-016d-4d9e-80d5-2ff88794dcdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:59.934572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:30:59.934572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:30:59.934572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:30:59.934572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:30:59.934572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:30:59.934572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:30:59.934572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:30:59.934572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:30:59.934572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:30:59.934572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:30:59.934572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:30:59.934572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:30:59.934572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:30:59.934572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:30:59.936292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:30:59.937276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:30:59.937276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:30:59.937276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:30:59.937276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:30:59.937276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 372/759] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:30:59.937276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd181e-016d-4d9e-80d5-2ff88794dcdf None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:30:59.948755 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7dd3d-ad84-4262-a733-718c33c7f758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:30:59.949638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7dd3d-ad84-4262-a733-718c33c7f758 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:30:59.950079 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7dd3d-ad84-4262-a733-718c33c7f758 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:30:59.950532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7dd3d-ad84-4262-a733-718c33c7f758 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:30:59.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7dd3d-ad84-4262-a733-718c33c7f758 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:30:59.953852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7dd3d-ad84-4262-a733-718c33c7f758 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:30:59.971726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 388/760] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 3 10:30:59 2022] POST /identity/v3/services => generated 240 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:30:59.972677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7dd3d-ad84-4262-a733-718c33c7f758 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:00.050449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:00.109057 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd181e-016d-4d9e-80d5-2ff88794dcdf None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:00.120561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7dd3d-ad84-4262-a733-718c33c7f758 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:00.167339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4beb73-4abc-473b-8b93-c19fcc840133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:00.168513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4beb73-4abc-473b-8b93-c19fcc840133 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:00.169008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4beb73-4abc-473b-8b93-c19fcc840133 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:00.169531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4beb73-4abc-473b-8b93-c19fcc840133 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:00.170150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca65411-6511-43ec-9cab-07e0382ef766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:00.170795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca65411-6511-43ec-9cab-07e0382ef766 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:00.171130 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca65411-6511-43ec-9cab-07e0382ef766 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:00.171489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca65411-6511-43ec-9cab-07e0382ef766 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:00.172352 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca65411-6511-43ec-9cab-07e0382ef766 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:00.174797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca65411-6511-43ec-9cab-07e0382ef766 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:00.182116 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 389/761] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 3 10:30:59 2022] GET /identity/v3/services => generated 772 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:00.185749 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4beb73-4abc-473b-8b93-c19fcc840133 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:00.186838 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca65411-6511-43ec-9cab-07e0382ef766 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:00.188547 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4beb73-4abc-473b-8b93-c19fcc840133 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:00.215004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-ca4beb73-4abc-473b-8b93-c19fcc840133 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:00.215004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:00.215004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:00.215004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:00.215004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:00.215004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:00.215004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:00.215004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:00.215004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:00.215004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:00.215004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:00.215004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:00.215004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:31:00.215004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:31:00.215004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:00.215985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:00.216887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:31:00.216887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:31:00.216887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:31:00.216887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:31:00.216887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:31:00.216887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:31:00.216887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:00.216887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:00.221697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 373/762] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:30:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:00.225526 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4beb73-4abc-473b-8b93-c19fcc840133 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:00.241683 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:00.373647 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca65411-6511-43ec-9cab-07e0382ef766 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:00.390062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4beb73-4abc-473b-8b93-c19fcc840133 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:00.413058 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:00.435018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224541a-7a68-4312-8d42-ab914a901861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:00.435659 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224541a-7a68-4312-8d42-ab914a901861 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:00.435982 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224541a-7a68-4312-8d42-ab914a901861 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:00.436397 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224541a-7a68-4312-8d42-ab914a901861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:00.437009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224541a-7a68-4312-8d42-ab914a901861 None admin] RBAC: Adding query filter params (role.id=1df33ad923a44a59b50dcacb6a75fefb, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=035ae78ef3cb43e985e06cc3fa971c8b) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:00.437324 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224541a-7a68-4312-8d42-ab914a901861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:00.438817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224541a-7a68-4312-8d42-ab914a901861 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:00.447934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7a786-5820-4b64-82a5-e78a3ee20756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:00.448525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7a786-5820-4b64-82a5-e78a3ee20756 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:00.448750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 390/763] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:31:00 2022] GET /identity/v3/role_assignments?user.id=035ae78ef3cb43e985e06cc3fa971c8b&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=1df33ad923a44a59b50dcacb6a75fefb => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:00.448852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7a786-5820-4b64-82a5-e78a3ee20756 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:00.449111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7a786-5820-4b64-82a5-e78a3ee20756 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:00.454578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7a786-5820-4b64-82a5-e78a3ee20756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:00.454578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7a786-5820-4b64-82a5-e78a3ee20756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:00.454578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7a786-5820-4b64-82a5-e78a3ee20756 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:00.454906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224541a-7a68-4312-8d42-ab914a901861 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:00.477447 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 374/764] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:00.483343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7a786-5820-4b64-82a5-e78a3ee20756 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:00.625601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:00.670382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224541a-7a68-4312-8d42-ab914a901861 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:00.687563 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7a786-5820-4b64-82a5-e78a3ee20756 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:00.714016 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f61d17-bf0d-463c-9155-89c5ffa8e3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:00.714926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f61d17-bf0d-463c-9155-89c5ffa8e3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:00.715212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f61d17-bf0d-463c-9155-89c5ffa8e3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:00.715490 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f61d17-bf0d-463c-9155-89c5ffa8e3f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:00.716258 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f61d17-bf0d-463c-9155-89c5ffa8e3f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:00.716580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f61d17-bf0d-463c-9155-89c5ffa8e3f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:00.718060 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f61d17-bf0d-463c-9155-89c5ffa8e3f2 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:00.730341 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc76b2-72c8-4b7d-947d-5f4f7900562e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:00.731009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc76b2-72c8-4b7d-947d-5f4f7900562e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:00.731287 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc76b2-72c8-4b7d-947d-5f4f7900562e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:00.731558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc76b2-72c8-4b7d-947d-5f4f7900562e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:00.732352 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 391/765] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:31:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:00.738417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f61d17-bf0d-463c-9155-89c5ffa8e3f2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:00.740709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc76b2-72c8-4b7d-947d-5f4f7900562e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:00.742205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc76b2-72c8-4b7d-947d-5f4f7900562e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-33fc76b2-72c8-4b7d-947d-5f4f7900562e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:31:00.755928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:00.756956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:00.757942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:00.757942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:00.757942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:31:00.757942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:31:00.757942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:31:00.757942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:31:00.757942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:31:00.757942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:31:00.757942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:31:00.757942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:00.757942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 375/766] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 3 10:31:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:00.763485 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc76b2-72c8-4b7d-947d-5f4f7900562e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:00.797635 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:00.859623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f61d17-bf0d-463c-9155-89c5ffa8e3f2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:00.884447 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc76b2-72c8-4b7d-947d-5f4f7900562e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:00.905839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505f24b-41a2-4a4b-87e3-c974d877d83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:00.906888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505f24b-41a2-4a4b-87e3-c974d877d83a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:00.907390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505f24b-41a2-4a4b-87e3-c974d877d83a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:00.908035 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505f24b-41a2-4a4b-87e3-c974d877d83a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:00.909726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505f24b-41a2-4a4b-87e3-c974d877d83a None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=a81ea5afe74b43879abc94b3cd134fd2) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:00.910277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505f24b-41a2-4a4b-87e3-c974d877d83a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:00.912013 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505f24b-41a2-4a4b-87e3-c974d877d83a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:00.919751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 392/767] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:31:00 2022] GET /identity/v3/role_assignments?user.id=a81ea5afe74b43879abc94b3cd134fd2&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=01714797cab143e1a7a150c068fa9315 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:00.926000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:00.943451 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f479a-59c3-43f8-b652-7fc08e03f525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:00.944134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f479a-59c3-43f8-b652-7fc08e03f525 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:00.944424 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f479a-59c3-43f8-b652-7fc08e03f525 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:00.944701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f479a-59c3-43f8-b652-7fc08e03f525 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:00.945576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f479a-59c3-43f8-b652-7fc08e03f525 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:00.945892 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f479a-59c3-43f8-b652-7fc08e03f525 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:00.947443 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f479a-59c3-43f8-b652-7fc08e03f525 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:00.992296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 376/768] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 3 10:31:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:00.998421 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505f24b-41a2-4a4b-87e3-c974d877d83a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:01.167085 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505f24b-41a2-4a4b-87e3-c974d877d83a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:01.179285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:01.213107 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626e617-91a9-4bfd-baaa-106383c084e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:01.214465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626e617-91a9-4bfd-baaa-106383c084e7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:01.214958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626e617-91a9-4bfd-baaa-106383c084e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:01.215454 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626e617-91a9-4bfd-baaa-106383c084e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:01.228475 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626e617-91a9-4bfd-baaa-106383c084e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:01.230343 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626e617-91a9-4bfd-baaa-106383c084e7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:01.246215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-2626e617-91a9-4bfd-baaa-106383c084e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:01.246215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:01.246215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:01.246215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:01.246215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:01.246215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:01.246215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:01.246215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:01.246215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:01.246215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:01.246215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:01.246215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:01.246215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:01.246215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:01.246215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:01.247153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:01.248049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:01.248049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:01.248049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:01.248049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:01.250321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 393/769] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:01.262636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f479a-59c3-43f8-b652-7fc08e03f525 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:01.420193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f479a-59c3-43f8-b652-7fc08e03f525 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:01.476955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fae6f-fccc-4ac7-97c5-04360480cc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:01.477547 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fae6f-fccc-4ac7-97c5-04360480cc63 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:01.477832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fae6f-fccc-4ac7-97c5-04360480cc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:01.478104 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fae6f-fccc-4ac7-97c5-04360480cc63 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:01.478821 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5fae6f-fccc-4ac7-97c5-04360480cc63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:01.479124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5fae6f-fccc-4ac7-97c5-04360480cc63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:01.480633 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5fae6f-fccc-4ac7-97c5-04360480cc63 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:01.494569 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 377/770] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:01.500352 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626e617-91a9-4bfd-baaa-106383c084e7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:01.637250 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626e617-91a9-4bfd-baaa-106383c084e7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:01.679149 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c01fe-3a81-4a34-8146-fb3544bc8f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:01.680173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c01fe-3a81-4a34-8146-fb3544bc8f94 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:01.680699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c01fe-3a81-4a34-8146-fb3544bc8f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:01.681254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c01fe-3a81-4a34-8146-fb3544bc8f94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:01.689131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c01fe-3a81-4a34-8146-fb3544bc8f94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:01.691301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c01fe-3a81-4a34-8146-fb3544bc8f94 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:01.708200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-497c01fe-3a81-4a34-8146-fb3544bc8f94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:01.708200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:01.708200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:01.708200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:01.708200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:01.708200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:01.708200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:01.708200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:01.708200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:01.708200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:01.708200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:01.708200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:01.708200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:31:01.708200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:31:01.708200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:01.709153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:01.710067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:31:01.710067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:31:01.710067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:31:01.710067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:31:01.710067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:31:01.710067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:31:01.710067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:01.710067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:01.713879 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 394/771] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:01.719954 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fae6f-fccc-4ac7-97c5-04360480cc63 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:01.804716 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:01.899579 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fae6f-fccc-4ac7-97c5-04360480cc63 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:01.958388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4aeea-e929-4c88-a8c9-7247566ed4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:01.959392 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4aeea-e929-4c88-a8c9-7247566ed4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:01.959926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4aeea-e929-4c88-a8c9-7247566ed4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:01.960402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4aeea-e929-4c88-a8c9-7247566ed4b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:01.961376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4aeea-e929-4c88-a8c9-7247566ed4b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:01.961991 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4aeea-e929-4c88-a8c9-7247566ed4b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:01.964053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4aeea-e929-4c88-a8c9-7247566ed4b2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:01.987964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 378/772] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:31:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:01.997258 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c01fe-3a81-4a34-8146-fb3544bc8f94 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:02.031707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:02.118737 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c01fe-3a81-4a34-8146-fb3544bc8f94 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:02.161974 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604f207-64a0-4c66-9622-0fe95f2a533b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:02.162695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604f207-64a0-4c66-9622-0fe95f2a533b None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:02.162983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604f207-64a0-4c66-9622-0fe95f2a533b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:02.163267 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604f207-64a0-4c66-9622-0fe95f2a533b None admin] PATH_INFO: `/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/e66f25cc329f418eba6bd637bf3e8a3a/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:02.165546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604f207-64a0-4c66-9622-0fe95f2a533b None admin] RBAC: Authorizing `identity:create_grant(project_id=4272b859979d49c8b8f0c17ec9eb81bc, user_id=e66f25cc329f418eba6bd637bf3e8a3a, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:02.167171 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604f207-64a0-4c66-9622-0fe95f2a533b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:02.192080 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 395/773] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:31:01 2022] PUT /identity/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/e66f25cc329f418eba6bd637bf3e8a3a/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:31:02.626568 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb532b0-b687-49f0-94be-995caedacf59 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:02.627125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb532b0-b687-49f0-94be-995caedacf59 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:02.627125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb532b0-b687-49f0-94be-995caedacf59 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:02.628760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 379/774] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:02.634085 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d1d82-d0a7-4723-9d18-9e60712ac94c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:02.634540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d1d82-d0a7-4723-9d18-9e60712ac94c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:02.634717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d1d82-d0a7-4723-9d18-9e60712ac94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:02.656830 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-847d1d82-d0a7-4723-9d18-9e60712ac94c None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:02.658955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-847d1d82-d0a7-4723-9d18-9e60712ac94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:02.764011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 396/775] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:02.797944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb532b0-b687-49f0-94be-995caedacf59 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:02.927478 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb532b0-b687-49f0-94be-995caedacf59 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:02.939594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:02.980932 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ca9d9-2c71-4054-b0c5-4517484c61b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:02.981664 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ca9d9-2c71-4054-b0c5-4517484c61b7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:02.982048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ca9d9-2c71-4054-b0c5-4517484c61b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:02.982329 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ca9d9-2c71-4054-b0c5-4517484c61b7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:02.982963 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ca9d9-2c71-4054-b0c5-4517484c61b7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:02.984324 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ca9d9-2c71-4054-b0c5-4517484c61b7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-102ca9d9-2c71-4054-b0c5-4517484c61b7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:02.997051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:02.998081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:02.999525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:02.999525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:02.999525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:02.999525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:02.999525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:02.999525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:02.999525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:02.999525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:02.999525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:02.999525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:02.999525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:02.999525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:02.999525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 10:31:02.999525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:02.999525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 380/776] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:02 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:03.004278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d1d82-d0a7-4723-9d18-9e60712ac94c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:03.028423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:03.125232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d1d82-d0a7-4723-9d18-9e60712ac94c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:03.135045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc7de1-bbb5-43cd-9d52-5410550b9bb9 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:03.135383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc7de1-bbb5-43cd-9d52-5410550b9bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:03.135604 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc7de1-bbb5-43cd-9d52-5410550b9bb9 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:03.137264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 381/777] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:03.143278 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8406bc-90de-402a-b7ee-c90fea8d7bd2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:03.143841 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8406bc-90de-402a-b7ee-c90fea8d7bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:03.144320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8406bc-90de-402a-b7ee-c90fea8d7bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:03.170105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07af718-fbc1-4e2f-b692-24fc1187851a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:03.170748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07af718-fbc1-4e2f-b692-24fc1187851a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:03.171032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07af718-fbc1-4e2f-b692-24fc1187851a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:03.171309 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07af718-fbc1-4e2f-b692-24fc1187851a None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:03.172010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07af718-fbc1-4e2f-b692-24fc1187851a None admin] RBAC: Adding query filter params (name=network) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:03.172316 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07af718-fbc1-4e2f-b692-24fc1187851a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:03.173812 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07af718-fbc1-4e2f-b692-24fc1187851a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:03.182681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-6e8406bc-90de-402a-b7ee-c90fea8d7bd2 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:03.184172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6e8406bc-90de-402a-b7ee-c90fea8d7bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:03.189337 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 397/778] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 3 10:31:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:03.195344 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07af718-fbc1-4e2f-b692-24fc1187851a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:03.297368 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 382/779] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:03.299273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a91982-a3c2-41d5-9a10-526e00fe7c52 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:03.299775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a91982-a3c2-41d5-9a10-526e00fe7c52 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:03.300254 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a91982-a3c2-41d5-9a10-526e00fe7c52 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:03.302341 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 383/780] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:03.308453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b53943-ef68-4584-9b88-32a052c08bb2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:03.308733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b53943-ef68-4584-9b88-32a052c08bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:03.309012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b53943-ef68-4584-9b88-32a052c08bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:03.320650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:03.338832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e3b53943-ef68-4584-9b88-32a052c08bb2 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:03.340672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e3b53943-ef68-4584-9b88-32a052c08bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:03.354535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07af718-fbc1-4e2f-b692-24fc1187851a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:03.395819 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf245b5-962e-4239-9a9c-fc7ff67c1169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:03.396499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf245b5-962e-4239-9a9c-fc7ff67c1169 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:03.396787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf245b5-962e-4239-9a9c-fc7ff67c1169 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:03.397065 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf245b5-962e-4239-9a9c-fc7ff67c1169 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:03.397839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf245b5-962e-4239-9a9c-fc7ff67c1169 None admin] RBAC: Adding query filter params (type=network) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:03.398155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf245b5-962e-4239-9a9c-fc7ff67c1169 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:03.399520 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf245b5-962e-4239-9a9c-fc7ff67c1169 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:03.405558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 398/781] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 3 10:31:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:03.406569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf245b5-962e-4239-9a9c-fc7ff67c1169 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:03.439883 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 384/782] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:03.441180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b53943-ef68-4584-9b88-32a052c08bb2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:03.476395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:03.543312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf245b5-962e-4239-9a9c-fc7ff67c1169 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:03.561751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b53943-ef68-4584-9b88-32a052c08bb2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:03.583709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436defd4-e998-4ded-a2cd-2065c81dfa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:03.584365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436defd4-e998-4ded-a2cd-2065c81dfa74 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:03.584652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436defd4-e998-4ded-a2cd-2065c81dfa74 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:03.584926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436defd4-e998-4ded-a2cd-2065c81dfa74 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:03.585587 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436defd4-e998-4ded-a2cd-2065c81dfa74 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:03.587087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436defd4-e998-4ded-a2cd-2065c81dfa74 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-436defd4-e998-4ded-a2cd-2065c81dfa74 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:03.597283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:03.598522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:03.599381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 399/783] 10.222.0.47 () {56 vars in 1129 bytes} [Wed Aug 3 10:31:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:03.600807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436defd4-e998-4ded-a2cd-2065c81dfa74 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:03.604736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:03.605780 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ef4dc-8336-4fd1-a738-bb2d90387eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:03.606619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ef4dc-8336-4fd1-a738-bb2d90387eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:03.606929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ef4dc-8336-4fd1-a738-bb2d90387eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:03.607232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ef4dc-8336-4fd1-a738-bb2d90387eb5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:03.608062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ef4dc-8336-4fd1-a738-bb2d90387eb5 None admin] RBAC: Adding query filter params (interface=public, service_id=70eab5353a5d4e99a9f37ca2bf638da0, region_id=RegionOne) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:03.608383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ef4dc-8336-4fd1-a738-bb2d90387eb5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:03.609790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ef4dc-8336-4fd1-a738-bb2d90387eb5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:03.616709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 385/784] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 3 10:31:03 2022] GET /identity/v3/endpoints?service_id=70eab5353a5d4e99a9f37ca2bf638da0&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:03.617985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ed385-6fcf-4b76-a751-a1c97a736453 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:03.618283 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ed385-6fcf-4b76-a751-a1c97a736453 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:03.618565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ed385-6fcf-4b76-a751-a1c97a736453 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:03.619946 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 386/785] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:03.620588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ed385-6fcf-4b76-a751-a1c97a736453 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:03.727182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436defd4-e998-4ded-a2cd-2065c81dfa74 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:03.742081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:03.752986 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ed385-6fcf-4b76-a751-a1c97a736453 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:03.779438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc93b67-432a-42db-9874-e25575ab7252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:03.780389 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc93b67-432a-42db-9874-e25575ab7252 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:03.780895 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc93b67-432a-42db-9874-e25575ab7252 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:03.781401 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc93b67-432a-42db-9874-e25575ab7252 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:03.790611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82bca5-5342-47c7-80ed-b5b8ea5c0578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:03.791274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82bca5-5342-47c7-80ed-b5b8ea5c0578 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:03.791562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82bca5-5342-47c7-80ed-b5b8ea5c0578 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:03.791833 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82bca5-5342-47c7-80ed-b5b8ea5c0578 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:03.792534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82bca5-5342-47c7-80ed-b5b8ea5c0578 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:03.792847 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82bca5-5342-47c7-80ed-b5b8ea5c0578 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:03.794319 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82bca5-5342-47c7-80ed-b5b8ea5c0578 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:03.800522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc93b67-432a-42db-9874-e25575ab7252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:31:03.807311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 387/786] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Aug 3 10:31:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:03.807311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82bca5-5342-47c7-80ed-b5b8ea5c0578 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-7dc93b67-432a-42db-9874-e25575ab7252 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:31:03.807544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:03.808430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:03.809255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:03.809255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:03.809255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:03.809255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:03.809255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:31:03.809255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:31:03.809255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:31:03.809255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:31:03.809255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:31:03.809255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:31:03.809255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:31:03.809255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:03.812295 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 400/787] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:31:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:03.813354 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926bf46-20e7-4007-9c36-4e9fd2a1f173 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:03.813673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926bf46-20e7-4007-9c36-4e9fd2a1f173 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:03.813957 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926bf46-20e7-4007-9c36-4e9fd2a1f173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:03.834176 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e926bf46-20e7-4007-9c36-4e9fd2a1f173 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:03.835385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e926bf46-20e7-4007-9c36-4e9fd2a1f173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:03.903230 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:03.913082 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 401/788] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:03.913964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926bf46-20e7-4007-9c36-4e9fd2a1f173 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:03.938596 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82bca5-5342-47c7-80ed-b5b8ea5c0578 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:03.977664 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17012a4c-5ca8-41e8-bb30-8c039a5a2283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:03.978343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17012a4c-5ca8-41e8-bb30-8c039a5a2283 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:03.978704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17012a4c-5ca8-41e8-bb30-8c039a5a2283 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:03.979085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17012a4c-5ca8-41e8-bb30-8c039a5a2283 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:03.979876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17012a4c-5ca8-41e8-bb30-8c039a5a2283 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:03.981194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17012a4c-5ca8-41e8-bb30-8c039a5a2283 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:03.987621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 388/789] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 3 10:31:03 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:03.988502 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17012a4c-5ca8-41e8-bb30-8c039a5a2283 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:04.028966 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:04.071622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926bf46-20e7-4007-9c36-4e9fd2a1f173 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:04.115964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c65af1-f752-4dff-8914-c57c06458f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:04.116719 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c65af1-f752-4dff-8914-c57c06458f9b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:04.117005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c65af1-f752-4dff-8914-c57c06458f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:04.117279 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c65af1-f752-4dff-8914-c57c06458f9b None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:04.118032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c65af1-f752-4dff-8914-c57c06458f9b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:04.118403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c65af1-f752-4dff-8914-c57c06458f9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:04.119942 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c65af1-f752-4dff-8914-c57c06458f9b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:04.129934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 402/790] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Aug 3 10:31:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:04.130993 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c65af1-f752-4dff-8914-c57c06458f9b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:04.138581 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17012a4c-5ca8-41e8-bb30-8c039a5a2283 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:04.178699 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f32e6b-3fa4-459f-8fd5-511500e6e39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:04.179485 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f32e6b-3fa4-459f-8fd5-511500e6e39a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:04.179776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f32e6b-3fa4-459f-8fd5-511500e6e39a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:04.180116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f32e6b-3fa4-459f-8fd5-511500e6e39a None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:04.180955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f32e6b-3fa4-459f-8fd5-511500e6e39a None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:04.181279 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f32e6b-3fa4-459f-8fd5-511500e6e39a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:04.182840 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f32e6b-3fa4-459f-8fd5-511500e6e39a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:04.193605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 389/791] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:31:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:04.194872 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f32e6b-3fa4-459f-8fd5-511500e6e39a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:04.198459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:04.251259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c65af1-f752-4dff-8914-c57c06458f9b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:04.299704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92567269-d595-4239-bfc4-ed11966f2c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:04.301167 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92567269-d595-4239-bfc4-ed11966f2c2b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:04.301635 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92567269-d595-4239-bfc4-ed11966f2c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:04.302060 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92567269-d595-4239-bfc4-ed11966f2c2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:04.314720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:04.317335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92567269-d595-4239-bfc4-ed11966f2c2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-92567269-d595-4239-bfc4-ed11966f2c2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:31:04.323502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:04.324680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:04.325823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:04.325823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:04.325823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:04.325823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:04.325823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:31:04.325823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:31:04.325823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:31:04.325823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:31:04.325823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:31:04.325823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:31:04.325823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:31:04.325823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:04.328652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 403/792] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:31:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:04.329668 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92567269-d595-4239-bfc4-ed11966f2c2b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:04.348800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f32e6b-3fa4-459f-8fd5-511500e6e39a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:04.404061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68bd41-033a-4033-bbb4-202572639cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:04.405265 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68bd41-033a-4033-bbb4-202572639cb7 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:04.405829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68bd41-033a-4033-bbb4-202572639cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:04.406382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68bd41-033a-4033-bbb4-202572639cb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:04.408009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68bd41-033a-4033-bbb4-202572639cb7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:04.411115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68bd41-033a-4033-bbb4-202572639cb7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:04.452688 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 390/793] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:31:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:31:04.454551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68bd41-033a-4033-bbb4-202572639cb7 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:04.468523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92567269-d595-4239-bfc4-ed11966f2c2b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:04.491487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:04.523078 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403c3de-fef6-4c44-b397-48d7aa1f6cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:04.523660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403c3de-fef6-4c44-b397-48d7aa1f6cfa None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:04.523944 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403c3de-fef6-4c44-b397-48d7aa1f6cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:04.524222 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403c3de-fef6-4c44-b397-48d7aa1f6cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:04.537019 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403c3de-fef6-4c44-b397-48d7aa1f6cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:04.538685 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403c3de-fef6-4c44-b397-48d7aa1f6cfa None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:04.557906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-5403c3de-fef6-4c44-b397-48d7aa1f6cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:04.557906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:04.557906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:04.557906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:04.557906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:04.557906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:04.557906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:04.557906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:04.557906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:04.557906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:04.557906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:04.557906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:04.557906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:04.557906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:04.557906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:04.558924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:04.559831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:04.559831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:04.559831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:04.559831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:04.563881 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 404/794] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:04.565206 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403c3de-fef6-4c44-b397-48d7aa1f6cfa None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:04.614574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68bd41-033a-4033-bbb4-202572639cb7 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:04.663783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01423087-2599-48fd-9173-c401ac72a400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:04.664777 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01423087-2599-48fd-9173-c401ac72a400 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:04.665255 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01423087-2599-48fd-9173-c401ac72a400 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:04.666564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01423087-2599-48fd-9173-c401ac72a400 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:04.667605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01423087-2599-48fd-9173-c401ac72a400 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:04.668127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01423087-2599-48fd-9173-c401ac72a400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:04.671273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01423087-2599-48fd-9173-c401ac72a400 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:04.684113 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 391/795] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:31:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:04.695431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01423087-2599-48fd-9173-c401ac72a400 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:04.699651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:04.714424 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403c3de-fef6-4c44-b397-48d7aa1f6cfa None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:04.763614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e826343-9fa6-4003-92be-515f0ce033f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:04.764259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e826343-9fa6-4003-92be-515f0ce033f3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:04.764517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e826343-9fa6-4003-92be-515f0ce033f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:04.764789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e826343-9fa6-4003-92be-515f0ce033f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:04.765611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e826343-9fa6-4003-92be-515f0ce033f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:04.765924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e826343-9fa6-4003-92be-515f0ce033f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:04.767342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e826343-9fa6-4003-92be-515f0ce033f3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:04.789015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 405/796] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:04.789015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1806f3a-278c-4fb5-9b97-f101d4401807 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:04.789015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1806f3a-278c-4fb5-9b97-f101d4401807 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:04.789371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1806f3a-278c-4fb5-9b97-f101d4401807 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:04.790699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 406/797] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:04.791409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1806f3a-278c-4fb5-9b97-f101d4401807 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:04.835872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:04.854910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01423087-2599-48fd-9173-c401ac72a400 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:04.901101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535efc7-6596-49d0-a561-81bd5aaeb16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:04.901912 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535efc7-6596-49d0-a561-81bd5aaeb16a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:04.902196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535efc7-6596-49d0-a561-81bd5aaeb16a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:04.902495 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535efc7-6596-49d0-a561-81bd5aaeb16a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:04.918050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535efc7-6596-49d0-a561-81bd5aaeb16a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:04.920226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535efc7-6596-49d0-a561-81bd5aaeb16a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:04.927568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1806f3a-278c-4fb5-9b97-f101d4401807 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:04.938756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-c535efc7-6596-49d0-a561-81bd5aaeb16a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:04.938756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:04.938756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:04.938756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:04.938756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:04.938756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:04.938756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:04.938756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:04.938756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:04.938756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:04.938756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:04.938756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:04.938756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:04.938756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:04.938756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:04.940429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:04.941781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:04.941781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:04.941781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:04.941781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:04.941781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 392/798] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:04.945607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628eddab-2ba7-48de-b637-9fb7f523f42d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:04.945607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628eddab-2ba7-48de-b637-9fb7f523f42d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:04.945607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628eddab-2ba7-48de-b637-9fb7f523f42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:04.970778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-628eddab-2ba7-48de-b637-9fb7f523f42d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:04.972345 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-628eddab-2ba7-48de-b637-9fb7f523f42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:04.980865 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af208e2-819b-4644-8492-037476fbe143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:04.981692 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af208e2-819b-4644-8492-037476fbe143 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:04.982088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af208e2-819b-4644-8492-037476fbe143 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:04.982901 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af208e2-819b-4644-8492-037476fbe143 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:04.992494 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af208e2-819b-4644-8492-037476fbe143 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:04.994126 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af208e2-819b-4644-8492-037476fbe143 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-0af208e2-819b-4644-8492-037476fbe143 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:31:05.007053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:05.008499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:05.010715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:05.010715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:05.010715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:31:05.010715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:31:05.010715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:31:05.010715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:31:05.010715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:31:05.010715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:31:05.010715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 10:31:05.010715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:05.010715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 407/799] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 3 10:31:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:05.012175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af208e2-819b-4644-8492-037476fbe143 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:05.041520 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:05.067756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 393/800] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:05.068937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628eddab-2ba7-48de-b637-9fb7f523f42d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:05.107096 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:05.125250 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af208e2-819b-4644-8492-037476fbe143 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:05.164857 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ee37d-6113-4a42-b053-5347c6a75cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:05.165328 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ee37d-6113-4a42-b053-5347c6a75cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:05.165651 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ee37d-6113-4a42-b053-5347c6a75cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:05.165927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ee37d-6113-4a42-b053-5347c6a75cb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:05.166680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ee37d-6113-4a42-b053-5347c6a75cb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:05.166924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ee37d-6113-4a42-b053-5347c6a75cb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:05.168308 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ee37d-6113-4a42-b053-5347c6a75cb9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:05.182696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 408/801] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:05.183693 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ee37d-6113-4a42-b053-5347c6a75cb9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:05.210269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628eddab-2ba7-48de-b637-9fb7f523f42d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:05.260300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff09f4-5482-457b-9e2f-166006b4de00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:05.261185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff09f4-5482-457b-9e2f-166006b4de00 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:05.261657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff09f4-5482-457b-9e2f-166006b4de00 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:05.262110 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff09f4-5482-457b-9e2f-166006b4de00 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:05.263319 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff09f4-5482-457b-9e2f-166006b4de00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:05.264357 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff09f4-5482-457b-9e2f-166006b4de00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:05.266317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff09f4-5482-457b-9e2f-166006b4de00 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:05.291365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 394/802] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Aug 3 10:31:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:05.293239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff09f4-5482-457b-9e2f-166006b4de00 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:05.299559 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ee37d-6113-4a42-b053-5347c6a75cb9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:05.336947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120b4f2-66f7-41b6-affe-ae07aeb65d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:05.337749 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120b4f2-66f7-41b6-affe-ae07aeb65d0d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:05.338104 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120b4f2-66f7-41b6-affe-ae07aeb65d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:05.338442 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120b4f2-66f7-41b6-affe-ae07aeb65d0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:05.350070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120b4f2-66f7-41b6-affe-ae07aeb65d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-b120b4f2-66f7-41b6-affe-ae07aeb65d0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:31:05.355051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:05.356190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:05.357165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:05.357165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:05.357165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:05.357165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:05.357165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:31:05.357165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:31:05.357165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:31:05.357165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:31:05.357165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:31:05.357165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:31:05.357165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:31:05.357165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:05.357908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:05.357997 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 409/803] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:31:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:05.357997 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120b4f2-66f7-41b6-affe-ae07aeb65d0d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:05.370930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:05.435202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff09f4-5482-457b-9e2f-166006b4de00 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:05.475564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71387f06-63e3-43fa-a4a5-a1a04ac34379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:05.476588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71387f06-63e3-43fa-a4a5-a1a04ac34379 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:05.476875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71387f06-63e3-43fa-a4a5-a1a04ac34379 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:05.477151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71387f06-63e3-43fa-a4a5-a1a04ac34379 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:05.482219 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120b4f2-66f7-41b6-affe-ae07aeb65d0d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:05.487278 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71387f06-63e3-43fa-a4a5-a1a04ac34379 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:05.488774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71387f06-63e3-43fa-a4a5-a1a04ac34379 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-71387f06-63e3-43fa-a4a5-a1a04ac34379 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:31:05.504554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:05.505543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:05.506534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:05.506534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:05.506534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:31:05.506534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:31:05.506534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:31:05.506534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:31:05.506534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:31:05.506534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:31:05.506534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 10:31:05.506534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:05.506534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 395/804] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:31:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:05.507135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71387f06-63e3-43fa-a4a5-a1a04ac34379 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:05.523321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd7a72-26b9-42d6-a045-f8c0b84f6482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:05.524287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd7a72-26b9-42d6-a045-f8c0b84f6482 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:05.524878 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd7a72-26b9-42d6-a045-f8c0b84f6482 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:05.525352 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd7a72-26b9-42d6-a045-f8c0b84f6482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:05.539721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd7a72-26b9-42d6-a045-f8c0b84f6482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:05.542880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd7a72-26b9-42d6-a045-f8c0b84f6482 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:05.558863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-14fd7a72-26b9-42d6-a045-f8c0b84f6482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:05.558863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:05.558863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:05.558863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:05.558863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:05.558863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:05.558863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:05.558863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:05.558863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:05.558863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:05.558863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:05.558863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:05.558863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:05.558863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:05.558863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:05.559753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:05.560655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:05.560655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:05.560655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:05.560655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:05.567887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 410/805] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:05.567887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd7a72-26b9-42d6-a045-f8c0b84f6482 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:05.630797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:05.637774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71387f06-63e3-43fa-a4a5-a1a04ac34379 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:05.639376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:05.681188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd7a72-26b9-42d6-a045-f8c0b84f6482 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:05.682556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4e36a-4e2b-4497-97b9-122bea07c1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:05.683189 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4e36a-4e2b-4497-97b9-122bea07c1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:05.683476 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4e36a-4e2b-4497-97b9-122bea07c1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:05.683754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4e36a-4e2b-4497-97b9-122bea07c1f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:05.684788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4e36a-4e2b-4497-97b9-122bea07c1f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:05.685095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4e36a-4e2b-4497-97b9-122bea07c1f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:05.687534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4e36a-4e2b-4497-97b9-122bea07c1f6 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:05.698546 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 396/806] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:31:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:05.699431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4e36a-4e2b-4497-97b9-122bea07c1f6 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:05.722088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5deeb0d-0e75-4d27-84ca-32c0a338ee9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:05.723044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5deeb0d-0e75-4d27-84ca-32c0a338ee9d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:05.723650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5deeb0d-0e75-4d27-84ca-32c0a338ee9d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:05.724137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5deeb0d-0e75-4d27-84ca-32c0a338ee9d None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:05.725195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5deeb0d-0e75-4d27-84ca-32c0a338ee9d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:05.725742 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5deeb0d-0e75-4d27-84ca-32c0a338ee9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:05.727538 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5deeb0d-0e75-4d27-84ca-32c0a338ee9d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:05.748282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 411/807] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Aug 3 10:31:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:05.749793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5deeb0d-0e75-4d27-84ca-32c0a338ee9d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:05.828021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4e36a-4e2b-4497-97b9-122bea07c1f6 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:05.876494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96679e4c-84ff-405b-8e86-25441cd7bc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:05.877176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96679e4c-84ff-405b-8e86-25441cd7bc36 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:05.877450 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96679e4c-84ff-405b-8e86-25441cd7bc36 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:05.877729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96679e4c-84ff-405b-8e86-25441cd7bc36 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:05.878428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96679e4c-84ff-405b-8e86-25441cd7bc36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:05.878731 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96679e4c-84ff-405b-8e86-25441cd7bc36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:05.879273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5deeb0d-0e75-4d27-84ca-32c0a338ee9d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:05.880106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96679e4c-84ff-405b-8e86-25441cd7bc36 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:05.900973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 397/808] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:05.900973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96679e4c-84ff-405b-8e86-25441cd7bc36 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:05.903964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:05.927152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cd7ce-8346-43d2-98e4-389cee5e4b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:05.928227 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cd7ce-8346-43d2-98e4-389cee5e4b5d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:05.928763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cd7ce-8346-43d2-98e4-389cee5e4b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:05.929235 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cd7ce-8346-43d2-98e4-389cee5e4b5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:05.930383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:05.943136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791cd7ce-8346-43d2-98e4-389cee5e4b5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:05.945049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791cd7ce-8346-43d2-98e4-389cee5e4b5d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:05.963329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-791cd7ce-8346-43d2-98e4-389cee5e4b5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:05.963329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:05.963329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:05.963329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:05.963329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:05.963329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:05.963329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:05.963329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:05.963329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:05.963329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:05.963329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:05.963329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:05.963329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:05.963329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:05.963329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:05.964521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:05.965415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:05.965415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:05.965415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:05.965415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:05.967711 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 412/809] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:05.968926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cd7ce-8346-43d2-98e4-389cee5e4b5d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:06.024465 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96679e4c-84ff-405b-8e86-25441cd7bc36 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:06.072720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71737a-a1b5-431d-8dc2-e93613410534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:06.073526 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71737a-a1b5-431d-8dc2-e93613410534 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:06.073923 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71737a-a1b5-431d-8dc2-e93613410534 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:06.074205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71737a-a1b5-431d-8dc2-e93613410534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:06.083470 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cd7ce-8346-43d2-98e4-389cee5e4b5d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:06.088121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71737a-a1b5-431d-8dc2-e93613410534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:06.089899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71737a-a1b5-431d-8dc2-e93613410534 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:06.110812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-cc71737a-a1b5-431d-8dc2-e93613410534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:06.110812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:06.110812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:06.110812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:06.110812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:06.110812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:06.110812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:06.110812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:06.110812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:06.110812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:06.110812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:06.110812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:06.110812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:06.110812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:06.110812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:06.111877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:06.112751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:06.112751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:06.112751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:06.112751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:06.112751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 398/810] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:06.113532 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71737a-a1b5-431d-8dc2-e93613410534 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:06.125454 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a2679-ee4c-4d19-959f-631a6152da0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:06.126571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a2679-ee4c-4d19-959f-631a6152da0f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:06.127843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a2679-ee4c-4d19-959f-631a6152da0f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:06.128348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a2679-ee4c-4d19-959f-631a6152da0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:06.136599 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a2679-ee4c-4d19-959f-631a6152da0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:06.138560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a2679-ee4c-4d19-959f-631a6152da0f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:06.152504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-d76a2679-ee4c-4d19-959f-631a6152da0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:06.152504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:06.152504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:06.152504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:06.152504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:06.152504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:06.152504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:06.152504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:06.152504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:06.152504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:06.152504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:06.152504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:06.152504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:31:06.152504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:31:06.152504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:06.153491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:06.154405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:31:06.154405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:31:06.154405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:31:06.154405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:31:06.154405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:31:06.154405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:31:06.154405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:06.154405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:06.161595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 413/811] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:06.162914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a2679-ee4c-4d19-959f-631a6152da0f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:06.200788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:06.211517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:06.244882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71737a-a1b5-431d-8dc2-e93613410534 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:06.290502 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45e7d0-243b-40a9-89e3-f53500d390f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:06.291231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45e7d0-243b-40a9-89e3-f53500d390f2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:06.291584 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45e7d0-243b-40a9-89e3-f53500d390f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:06.291933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45e7d0-243b-40a9-89e3-f53500d390f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:06.292124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a2679-ee4c-4d19-959f-631a6152da0f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:06.292767 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45e7d0-243b-40a9-89e3-f53500d390f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:06.293163 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45e7d0-243b-40a9-89e3-f53500d390f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:06.294661 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45e7d0-243b-40a9-89e3-f53500d390f2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:06.307741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 399/812] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:06.308772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45e7d0-243b-40a9-89e3-f53500d390f2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:06.337203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b39fc3-68c2-4ca1-98ca-2f60ec85e2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:06.338326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b39fc3-68c2-4ca1-98ca-2f60ec85e2bf None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:06.338839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b39fc3-68c2-4ca1-98ca-2f60ec85e2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:06.339910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b39fc3-68c2-4ca1-98ca-2f60ec85e2bf None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:06.341064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b39fc3-68c2-4ca1-98ca-2f60ec85e2bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:06.341742 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b39fc3-68c2-4ca1-98ca-2f60ec85e2bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:06.343912 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b39fc3-68c2-4ca1-98ca-2f60ec85e2bf None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:06.358791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 414/813] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:06.359911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b39fc3-68c2-4ca1-98ca-2f60ec85e2bf None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:06.417983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45e7d0-243b-40a9-89e3-f53500d390f2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:06.450501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:06.453571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff42f4-d7cf-4cb8-acc9-81e299984e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:06.454445 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff42f4-d7cf-4cb8-acc9-81e299984e74 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:06.454554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff42f4-d7cf-4cb8-acc9-81e299984e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:06.454788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff42f4-d7cf-4cb8-acc9-81e299984e74 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:06.455654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff42f4-d7cf-4cb8-acc9-81e299984e74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:06.456432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff42f4-d7cf-4cb8-acc9-81e299984e74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:06.457872 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff42f4-d7cf-4cb8-acc9-81e299984e74 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:06.460516 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:06.470941 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b39fc3-68c2-4ca1-98ca-2f60ec85e2bf None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:06.473376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 400/814] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:31:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:06.474733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3fe1b-bea1-48c0-8938-cdd3d6cb2e96 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:06.475025 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3fe1b-bea1-48c0-8938-cdd3d6cb2e96 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:06.475301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3fe1b-bea1-48c0-8938-cdd3d6cb2e96 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:06.480533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 401/815] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:06.480533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3fe1b-bea1-48c0-8938-cdd3d6cb2e96 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:06.515300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806d3c5-cbf8-4d0b-927f-163aae3ab180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:06.515987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d3c5-cbf8-4d0b-927f-163aae3ab180 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:06.516272 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d3c5-cbf8-4d0b-927f-163aae3ab180 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:06.516551 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d3c5-cbf8-4d0b-927f-163aae3ab180 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:06.524605 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806d3c5-cbf8-4d0b-927f-163aae3ab180 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:06.526098 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806d3c5-cbf8-4d0b-927f-163aae3ab180 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-e806d3c5-cbf8-4d0b-927f-163aae3ab180 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:31:06.538277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:06.539273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:06.540294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:06.540294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:06.540294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:31:06.540294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:31:06.540294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:31:06.540294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:31:06.540294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:31:06.540294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:31:06.540294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:31:06.540294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:06.540294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 415/816] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 3 10:31:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:06.540872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806d3c5-cbf8-4d0b-927f-163aae3ab180 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:06.590466 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3fe1b-bea1-48c0-8938-cdd3d6cb2e96 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:06.629146 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac929e7-5733-4ae3-827d-11423789629f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:06.629836 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac929e7-5733-4ae3-827d-11423789629f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:06.630125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac929e7-5733-4ae3-827d-11423789629f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:06.630401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac929e7-5733-4ae3-827d-11423789629f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:06.640879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac929e7-5733-4ae3-827d-11423789629f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:06.643099 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac929e7-5733-4ae3-827d-11423789629f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:06.643621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806d3c5-cbf8-4d0b-927f-163aae3ab180 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:06.660288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-8ac929e7-5733-4ae3-827d-11423789629f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:06.660288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:06.660288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:06.660288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:06.660288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:06.660288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:06.660288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:06.660288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:06.660288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:06.660288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:06.660288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:06.660288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:06.660288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:31:06.660288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:31:06.660288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:06.661224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:06.662153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:31:06.662153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:31:06.662153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:31:06.662153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:31:06.662153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:31:06.662153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:31:06.662153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:06.662153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:06.662581 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 402/817] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:06.663730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c38e9-2a15-49bd-9359-a2f5ecce6119 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:06.664020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c38e9-2a15-49bd-9359-a2f5ecce6119 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:06.664304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c38e9-2a15-49bd-9359-a2f5ecce6119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:06.686870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-6d6c38e9-2a15-49bd-9359-a2f5ecce6119 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:06.687011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05c638-2aa2-427e-896f-06e35d30acac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:06.687571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05c638-2aa2-427e-896f-06e35d30acac None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:06.687851 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05c638-2aa2-427e-896f-06e35d30acac None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:06.688065 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6d6c38e9-2a15-49bd-9359-a2f5ecce6119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:06.688164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05c638-2aa2-427e-896f-06e35d30acac None admin] PATH_INFO: `/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/035ae78ef3cb43e985e06cc3fa971c8b/roles/1df33ad923a44a59b50dcacb6a75fefb` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:06.705706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05c638-2aa2-427e-896f-06e35d30acac None admin] RBAC: Authorizing `identity:create_grant(project_id=4272b859979d49c8b8f0c17ec9eb81bc, user_id=035ae78ef3cb43e985e06cc3fa971c8b, role_id=1df33ad923a44a59b50dcacb6a75fefb)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:06.707163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05c638-2aa2-427e-896f-06e35d30acac None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:06.718193 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:06.725696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 416/818] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:31:06 2022] PUT /identity/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/035ae78ef3cb43e985e06cc3fa971c8b/roles/1df33ad923a44a59b50dcacb6a75fefb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:31:06.726567 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05c638-2aa2-427e-896f-06e35d30acac None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:06.769128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:06.792394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 403/819] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:06.793745 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c38e9-2a15-49bd-9359-a2f5ecce6119 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:06.860076 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05c638-2aa2-427e-896f-06e35d30acac None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:06.908806 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760e58c-cdd8-436c-ac8a-fbeb1361171c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:06.910150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760e58c-cdd8-436c-ac8a-fbeb1361171c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:06.910558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760e58c-cdd8-436c-ac8a-fbeb1361171c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:06.910805 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760e58c-cdd8-436c-ac8a-fbeb1361171c None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:06.912032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760e58c-cdd8-436c-ac8a-fbeb1361171c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:06.912621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760e58c-cdd8-436c-ac8a-fbeb1361171c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:06.914694 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760e58c-cdd8-436c-ac8a-fbeb1361171c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:06.925631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c38e9-2a15-49bd-9359-a2f5ecce6119 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:06.935074 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 417/820] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 3 10:31:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:06.936729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd7111-a243-482b-a9ff-0abe2cb61a0e None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:06.937018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd7111-a243-482b-a9ff-0abe2cb61a0e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:06.937294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd7111-a243-482b-a9ff-0abe2cb61a0e None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:06.940591 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 418/821] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:06.940591 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd7111-a243-482b-a9ff-0abe2cb61a0e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:06.972467 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c03e0-a60e-4bc9-a608-8cd7d77df4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:06.976530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c03e0-a60e-4bc9-a608-8cd7d77df4ad None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:06.977031 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c03e0-a60e-4bc9-a608-8cd7d77df4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:06.977517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c03e0-a60e-4bc9-a608-8cd7d77df4ad None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:06.978552 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c03e0-a60e-4bc9-a608-8cd7d77df4ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:06.979907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c03e0-a60e-4bc9-a608-8cd7d77df4ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:06.981692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c03e0-a60e-4bc9-a608-8cd7d77df4ad None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:07.000032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 404/822] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:31:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:07.000784 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c03e0-a60e-4bc9-a608-8cd7d77df4ad None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:07.009843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:07.054063 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:07.059888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd7111-a243-482b-a9ff-0abe2cb61a0e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:07.134878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c03e0-a60e-4bc9-a608-8cd7d77df4ad None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:07.177135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac5737-d106-4174-8a17-3c05a2e59887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:07.177809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac5737-d106-4174-8a17-3c05a2e59887 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:07.178095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac5737-d106-4174-8a17-3c05a2e59887 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:07.178382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac5737-d106-4174-8a17-3c05a2e59887 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:07.219273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992faa7-a0e2-4a27-b4c7-ea577f1c6ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:07.219781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992faa7-a0e2-4a27-b4c7-ea577f1c6ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:07.220064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992faa7-a0e2-4a27-b4c7-ea577f1c6ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:07.220334 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992faa7-a0e2-4a27-b4c7-ea577f1c6ee1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:07.221556 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992faa7-a0e2-4a27-b4c7-ea577f1c6ee1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:07.222938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992faa7-a0e2-4a27-b4c7-ea577f1c6ee1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-4992faa7-a0e2-4a27-b4c7-ea577f1c6ee1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:07.232496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:07.233601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:07.234458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:07.234458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:07.234458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:07.234458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:07.234458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:07.234458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:07.234458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:07.234458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:07.234458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:07.234458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:07.234458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:07.234458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:07.234458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 10:31:07.234458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:07.234458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 419/823] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:06 2022] GET /identity/v3/services/network => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:07.235286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc68f3-c4a6-4d76-90af-51f01ea9aea3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:07.235286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc68f3-c4a6-4d76-90af-51f01ea9aea3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:07.235417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc68f3-c4a6-4d76-90af-51f01ea9aea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:07.255412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-01cc68f3-c4a6-4d76-90af-51f01ea9aea3 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:07.256670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-01cc68f3-c4a6-4d76-90af-51f01ea9aea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:07.268884 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac5737-d106-4174-8a17-3c05a2e59887 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:07.270533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac5737-d106-4174-8a17-3c05a2e59887 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:07.325201 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:07.349500 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 420/824] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:07.349500 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc68f3-c4a6-4d76-90af-51f01ea9aea3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:07.400044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 405/825] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:31:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 10:31:07.401504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:07.407570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac5737-d106-4174-8a17-3c05a2e59887 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:07.463804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc68f3-c4a6-4d76-90af-51f01ea9aea3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:07.503536 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93439233-edde-47a6-a9a7-da73acf63d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:07.504226 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93439233-edde-47a6-a9a7-da73acf63d5d None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:07.504508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93439233-edde-47a6-a9a7-da73acf63d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:07.504789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93439233-edde-47a6-a9a7-da73acf63d5d None admin] PATH_INFO: `/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/a81ea5afe74b43879abc94b3cd134fd2/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:07.506907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93439233-edde-47a6-a9a7-da73acf63d5d None admin] RBAC: Authorizing `identity:create_grant(project_id=4272b859979d49c8b8f0c17ec9eb81bc, user_id=a81ea5afe74b43879abc94b3cd134fd2, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:07.508348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93439233-edde-47a6-a9a7-da73acf63d5d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:07.533793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 421/826] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:31:07 2022] PUT /identity/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/a81ea5afe74b43879abc94b3cd134fd2/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:31:07.533793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93439233-edde-47a6-a9a7-da73acf63d5d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:07.557426 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac5737-d106-4174-8a17-3c05a2e59887 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:07.647725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f242a5c-957b-4c3f-847c-dcc2b3016fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:07.648370 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f242a5c-957b-4c3f-847c-dcc2b3016fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:07.648652 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f242a5c-957b-4c3f-847c-dcc2b3016fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:07.648927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f242a5c-957b-4c3f-847c-dcc2b3016fe7 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:07.649651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f242a5c-957b-4c3f-847c-dcc2b3016fe7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:07.649961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f242a5c-957b-4c3f-847c-dcc2b3016fe7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:07.652316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f242a5c-957b-4c3f-847c-dcc2b3016fe7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:07.669301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 406/827] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 3 10:31:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:07.670358 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f242a5c-957b-4c3f-847c-dcc2b3016fe7 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:07.687311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:07.705718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93439233-edde-47a6-a9a7-da73acf63d5d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:07.730194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:07.776515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490148b-ea3e-4221-936b-56d6302ac9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:07.777637 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490148b-ea3e-4221-936b-56d6302ac9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:07.778078 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490148b-ea3e-4221-936b-56d6302ac9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:07.778525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490148b-ea3e-4221-936b-56d6302ac9d1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:07.779756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490148b-ea3e-4221-936b-56d6302ac9d1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:07.782303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490148b-ea3e-4221-936b-56d6302ac9d1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-0490148b-ea3e-4221-936b-56d6302ac9d1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:07.795131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:07.796386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:07.797322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:07.797322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:07.797322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:07.797322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:07.797322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:07.797322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:07.797322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:07.797322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:07.797322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:07.797322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:07.797322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:07.797322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:07.797322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 10:31:07.797322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:07.798229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f242a5c-957b-4c3f-847c-dcc2b3016fe7 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:07.799897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 422/828] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:31:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:07.800839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490148b-ea3e-4221-936b-56d6302ac9d1 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:07.838946 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0491f38-8275-4d9e-a4b3-9579f41f0ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:07.840107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0491f38-8275-4d9e-a4b3-9579f41f0ade None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:07.840531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0491f38-8275-4d9e-a4b3-9579f41f0ade None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:07.841012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0491f38-8275-4d9e-a4b3-9579f41f0ade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:07.852639 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0491f38-8275-4d9e-a4b3-9579f41f0ade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:07.854150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0491f38-8275-4d9e-a4b3-9579f41f0ade None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:07.869988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-d0491f38-8275-4d9e-a4b3-9579f41f0ade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:07.869988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:07.869988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:07.869988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:07.869988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:07.869988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:07.869988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:07.869988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:07.869988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:07.869988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:07.869988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:07.869988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:07.869988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:07.869988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:07.869988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:07.870927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:07.871826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:07.871826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:07.871826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:07.871826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:07.871826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 407/829] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:07.872554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0491f38-8275-4d9e-a4b3-9579f41f0ade None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:07.934684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490148b-ea3e-4221-936b-56d6302ac9d1 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:07.946575 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:07.982825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b9ab0-d4ae-49f2-9d5d-defa67de5b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:07.983521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b9ab0-d4ae-49f2-9d5d-defa67de5b60 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:07.983852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b9ab0-d4ae-49f2-9d5d-defa67de5b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:07.984134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b9ab0-d4ae-49f2-9d5d-defa67de5b60 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:07.984967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b9ab0-d4ae-49f2-9d5d-defa67de5b60 None admin] RBAC: Adding query filter params (type=network) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:07.985218 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b9ab0-d4ae-49f2-9d5d-defa67de5b60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:07.986771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b9ab0-d4ae-49f2-9d5d-defa67de5b60 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:07.992712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 423/830] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 3 10:31:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:07.993238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0491f38-8275-4d9e-a4b3-9579f41f0ade None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:07.993728 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b9ab0-d4ae-49f2-9d5d-defa67de5b60 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:08.037871 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:08.038940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de7949-4837-459a-8410-e99c5d7e093c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:08.039576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de7949-4837-459a-8410-e99c5d7e093c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:08.039858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de7949-4837-459a-8410-e99c5d7e093c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:08.040135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de7949-4837-459a-8410-e99c5d7e093c None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:08.040821 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de7949-4837-459a-8410-e99c5d7e093c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:08.041125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de7949-4837-459a-8410-e99c5d7e093c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:08.042558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de7949-4837-459a-8410-e99c5d7e093c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:08.052724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 408/831] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:31:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:08.052724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de7949-4837-459a-8410-e99c5d7e093c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:08.124867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b9ab0-d4ae-49f2-9d5d-defa67de5b60 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:08.163553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9b635-8161-4710-a8fc-a9bdf063e8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:08.164313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b635-8161-4710-a8fc-a9bdf063e8fe None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:08.164705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b635-8161-4710-a8fc-a9bdf063e8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:08.165086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b635-8161-4710-a8fc-a9bdf063e8fe None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:08.165914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9b635-8161-4710-a8fc-a9bdf063e8fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:08.166317 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9b635-8161-4710-a8fc-a9bdf063e8fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:08.168119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9b635-8161-4710-a8fc-a9bdf063e8fe None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:08.178751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de7949-4837-459a-8410-e99c5d7e093c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:08.180988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 424/832] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:08.182163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9b635-8161-4710-a8fc-a9bdf063e8fe None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:08.211794 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:08.223652 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8e994-be0b-4305-9b0f-10fa1fdbd67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:08.225837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8e994-be0b-4305-9b0f-10fa1fdbd67c None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:08.226123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8e994-be0b-4305-9b0f-10fa1fdbd67c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:08.226399 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8e994-be0b-4305-9b0f-10fa1fdbd67c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:08.227130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8e994-be0b-4305-9b0f-10fa1fdbd67c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:08.228522 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8e994-be0b-4305-9b0f-10fa1fdbd67c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:08.251535 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 409/833] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:31:08 2022] POST /identity/v3/endpoints => generated 337 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:31:08.253531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8e994-be0b-4305-9b0f-10fa1fdbd67c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:08.301734 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:08.306531 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9b635-8161-4710-a8fc-a9bdf063e8fe None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:08.350023 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559847e-f709-48f8-bd05-31933a3e42b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:08.350973 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559847e-f709-48f8-bd05-31933a3e42b4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:08.351459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559847e-f709-48f8-bd05-31933a3e42b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:08.351938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559847e-f709-48f8-bd05-31933a3e42b4 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:08.352846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559847e-f709-48f8-bd05-31933a3e42b4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:08.353365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559847e-f709-48f8-bd05-31933a3e42b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:08.355085 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559847e-f709-48f8-bd05-31933a3e42b4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:08.361026 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 425/834] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:31:08 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:08.379435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8e994-be0b-4305-9b0f-10fa1fdbd67c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:08.421160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e10293-a745-4fca-830b-366a03378332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:08.421889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e10293-a745-4fca-830b-366a03378332 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:08.422176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e10293-a745-4fca-830b-366a03378332 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:08.422450 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e10293-a745-4fca-830b-366a03378332 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:08.431290 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e10293-a745-4fca-830b-366a03378332 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:08.432783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e10293-a745-4fca-830b-366a03378332 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:08.446984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-88e10293-a745-4fca-830b-366a03378332 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:08.446984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:08.446984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:08.446984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:08.446984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:08.446984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:08.446984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:08.446984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:08.446984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:08.446984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:08.446984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:08.446984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:08.446984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:31:08.446984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:31:08.446984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:08.447929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:08.448815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:31:08.448815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:31:08.448815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:31:08.448815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:31:08.448815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:31:08.448815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:31:08.448815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:08.448815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:08.448815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 410/835] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:08.454100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559847e-f709-48f8-bd05-31933a3e42b4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:08.582671 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559847e-f709-48f8-bd05-31933a3e42b4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:08.618416 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6e409-d519-4e77-a5b6-f88277555deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:08.619037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6e409-d519-4e77-a5b6-f88277555deb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:08.619313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6e409-d519-4e77-a5b6-f88277555deb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:08.619588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6e409-d519-4e77-a5b6-f88277555deb None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:08.620351 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6e409-d519-4e77-a5b6-f88277555deb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:08.620657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6e409-d519-4e77-a5b6-f88277555deb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:08.622128 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6e409-d519-4e77-a5b6-f88277555deb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:08.635191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 426/836] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:31:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:08.641248 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e10293-a745-4fca-830b-366a03378332 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:08.714809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:08.775166 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e10293-a745-4fca-830b-366a03378332 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:08.816821 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc33c8-3faa-4f73-8238-31619aae4eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:08.817601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc33c8-3faa-4f73-8238-31619aae4eff None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:08.817887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc33c8-3faa-4f73-8238-31619aae4eff None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:08.818163 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc33c8-3faa-4f73-8238-31619aae4eff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:08.818964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc33c8-3faa-4f73-8238-31619aae4eff None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=e66f25cc329f418eba6bd637bf3e8a3a) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:08.819273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc33c8-3faa-4f73-8238-31619aae4eff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:08.820906 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc33c8-3faa-4f73-8238-31619aae4eff None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:08.827814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 411/837] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:31:08 2022] GET /identity/v3/role_assignments?user.id=e66f25cc329f418eba6bd637bf3e8a3a&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=01714797cab143e1a7a150c068fa9315 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:08.972473 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f733e-9919-4977-8abc-d3689c49684b None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:08.973004 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f733e-9919-4977-8abc-d3689c49684b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:08.973004 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f733e-9919-4977-8abc-d3689c49684b None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:08.974632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 427/838] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:08.980913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4c336-5721-4a8d-b0a3-d3ed3df192c4 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:08.981379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4c336-5721-4a8d-b0a3-d3ed3df192c4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:08.981456 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4c336-5721-4a8d-b0a3-d3ed3df192c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:09.007974 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-6bd4c336-5721-4a8d-b0a3-d3ed3df192c4 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:09.009181 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6bd4c336-5721-4a8d-b0a3-d3ed3df192c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:09.082534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 412/839] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:09.106016 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f733e-9919-4977-8abc-d3689c49684b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:09.126476 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:09.217612 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f733e-9919-4977-8abc-d3689c49684b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:09.254392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1091a-d24d-4442-9a2a-d496026ff242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:09.255014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1091a-d24d-4442-9a2a-d496026ff242 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:09.255295 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1091a-d24d-4442-9a2a-d496026ff242 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:09.255566 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1091a-d24d-4442-9a2a-d496026ff242 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:09.266309 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1091a-d24d-4442-9a2a-d496026ff242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-b0d1091a-d24d-4442-9a2a-d496026ff242 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:31:09.271695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:09.272634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:09.273489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:09.273489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:09.273489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:09.273489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:09.273489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:31:09.273489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:31:09.273489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:31:09.273489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:31:09.273489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:31:09.273489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:31:09.273489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:31:09.273489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:09.273489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 428/840] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:31:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:09.279265 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4c336-5721-4a8d-b0a3-d3ed3df192c4 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:09.317562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:09.386665 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4c336-5721-4a8d-b0a3-d3ed3df192c4 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:09.423293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73388d11-a933-4753-b1dd-dbeb16ca8cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:09.424827 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73388d11-a933-4753-b1dd-dbeb16ca8cbe None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:09.425286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73388d11-a933-4753-b1dd-dbeb16ca8cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:09.425759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73388d11-a933-4753-b1dd-dbeb16ca8cbe None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:09.426649 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73388d11-a933-4753-b1dd-dbeb16ca8cbe None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:09.427127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73388d11-a933-4753-b1dd-dbeb16ca8cbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:09.428684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73388d11-a933-4753-b1dd-dbeb16ca8cbe None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:09.437943 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 413/841] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:31:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:09.443742 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1091a-d24d-4442-9a2a-d496026ff242 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:09.533949 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:09.550954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1091a-d24d-4442-9a2a-d496026ff242 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:09.588057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c20ab-bbe0-4d88-8a0a-18de2c0bbd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:09.589057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c20ab-bbe0-4d88-8a0a-18de2c0bbd18 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:09.589859 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c20ab-bbe0-4d88-8a0a-18de2c0bbd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:09.590360 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c20ab-bbe0-4d88-8a0a-18de2c0bbd18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:09.602005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c20ab-bbe0-4d88-8a0a-18de2c0bbd18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:09.603776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c20ab-bbe0-4d88-8a0a-18de2c0bbd18 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:09.620164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ce9c20ab-bbe0-4d88-8a0a-18de2c0bbd18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:09.620164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:09.620164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:09.620164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:09.620164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:09.620164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:09.620164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:09.620164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:09.620164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:09.620164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:09.620164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:09.620164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:09.620164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:09.620164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:09.620164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:09.621195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:09.622135 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:09.622135 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:09.622135 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:09.622135 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:09.624211 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 429/842] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:09.630672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73388d11-a933-4753-b1dd-dbeb16ca8cbe None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:09.735908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:09.742717 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73388d11-a933-4753-b1dd-dbeb16ca8cbe None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:09.742886 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd244b24-b862-4c05-bf3c-240721507f2f None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:09.743117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd244b24-b862-4c05-bf3c-240721507f2f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:09.743411 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd244b24-b862-4c05-bf3c-240721507f2f None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:09.745108 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 430/843] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:09.750981 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d2e9d-292d-4a05-81bc-a73603acd43b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:09.751267 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d2e9d-292d-4a05-81bc-a73603acd43b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:09.751543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d2e9d-292d-4a05-81bc-a73603acd43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:09.770487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-447d2e9d-292d-4a05-81bc-a73603acd43b None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:09.771651 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-447d2e9d-292d-4a05-81bc-a73603acd43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:09.781543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4496ab-ade8-4cec-9801-80642a32d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:09.782415 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4496ab-ade8-4cec-9801-80642a32d74e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:09.782854 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4496ab-ade8-4cec-9801-80642a32d74e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:09.783307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4496ab-ade8-4cec-9801-80642a32d74e None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:09.784235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4496ab-ade8-4cec-9801-80642a32d74e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:09.784691 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4496ab-ade8-4cec-9801-80642a32d74e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:09.786231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4496ab-ade8-4cec-9801-80642a32d74e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:09.799884 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 414/844] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:09.806002 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4496ab-ade8-4cec-9801-80642a32d74e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:09.864174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 431/845] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:09.887112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d2e9d-292d-4a05-81bc-a73603acd43b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:09.941931 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4496ab-ade8-4cec-9801-80642a32d74e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:09.967434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:09.990995 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd18e26-5916-4ecb-8f88-2b1565d6dce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:09.991557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd18e26-5916-4ecb-8f88-2b1565d6dce4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:09.991788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd18e26-5916-4ecb-8f88-2b1565d6dce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:09.991974 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd18e26-5916-4ecb-8f88-2b1565d6dce4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:09.999851 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd18e26-5916-4ecb-8f88-2b1565d6dce4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:10.001296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd18e26-5916-4ecb-8f88-2b1565d6dce4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-5fd18e26-5916-4ecb-8f88-2b1565d6dce4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:31:10.014720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:10.015721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:10.016672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:10.016672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:10.016672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:31:10.016672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:31:10.016672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:31:10.016672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:31:10.016672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:31:10.016672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:31:10.016672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 10:31:10.016672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:10.020785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 415/846] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 3 10:31:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:10.021827 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd18e26-5916-4ecb-8f88-2b1565d6dce4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:10.027493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d2e9d-292d-4a05-81bc-a73603acd43b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:10.046935 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:10.076169 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1841b-e951-49d0-98c6-baf79ed95591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:10.077016 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1841b-e951-49d0-98c6-baf79ed95591 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:10.077299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1841b-e951-49d0-98c6-baf79ed95591 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:10.077607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1841b-e951-49d0-98c6-baf79ed95591 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:10.091338 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1841b-e951-49d0-98c6-baf79ed95591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-0fb1841b-e951-49d0-98c6-baf79ed95591 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:31:10.097667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:10.098640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:10.099496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:10.099496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:10.099496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:10.099496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:10.099496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:31:10.099496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:31:10.099496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:31:10.099496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:31:10.099496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:31:10.099496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:31:10.099496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:31:10.099496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:10.103846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 432/847] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:31:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:10.104786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1841b-e951-49d0-98c6-baf79ed95591 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:10.156642 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd18e26-5916-4ecb-8f88-2b1565d6dce4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:10.210016 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5af6f5-b83c-4b6e-9491-c4d4d8548a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:10.211411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5af6f5-b83c-4b6e-9491-c4d4d8548a4f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:10.212177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5af6f5-b83c-4b6e-9491-c4d4d8548a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:10.212662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5af6f5-b83c-4b6e-9491-c4d4d8548a4f None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:10.213973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5af6f5-b83c-4b6e-9491-c4d4d8548a4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:10.215320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5af6f5-b83c-4b6e-9491-c4d4d8548a4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:10.217556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5af6f5-b83c-4b6e-9491-c4d4d8548a4f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:10.240274 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1841b-e951-49d0-98c6-baf79ed95591 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:10.244849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 416/848] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Aug 3 10:31:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:10.250407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5af6f5-b83c-4b6e-9491-c4d4d8548a4f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:10.271428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:10.295483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d770c43-d169-40a1-bc90-c0b7f4d9ef5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:10.296607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d770c43-d169-40a1-bc90-c0b7f4d9ef5d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:10.297204 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d770c43-d169-40a1-bc90-c0b7f4d9ef5d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:10.297745 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d770c43-d169-40a1-bc90-c0b7f4d9ef5d None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:10.299915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d770c43-d169-40a1-bc90-c0b7f4d9ef5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:10.300449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d770c43-d169-40a1-bc90-c0b7f4d9ef5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:10.302586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d770c43-d169-40a1-bc90-c0b7f4d9ef5d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:10.315730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 433/849] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:31:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:10.321562 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d770c43-d169-40a1-bc90-c0b7f4d9ef5d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:10.354780 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:10.387293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5af6f5-b83c-4b6e-9491-c4d4d8548a4f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:10.433879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac013dc-b256-432a-a55a-5ac9bd184137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:10.435499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac013dc-b256-432a-a55a-5ac9bd184137 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:10.435988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac013dc-b256-432a-a55a-5ac9bd184137 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:10.436457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac013dc-b256-432a-a55a-5ac9bd184137 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:10.444622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d770c43-d169-40a1-bc90-c0b7f4d9ef5d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:10.485996 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4922be-6c28-4b2f-9f3f-886db4604e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:10.486925 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4922be-6c28-4b2f-9f3f-886db4604e01 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:10.487413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4922be-6c28-4b2f-9f3f-886db4604e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:10.487757 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4922be-6c28-4b2f-9f3f-886db4604e01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:10.501231 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4922be-6c28-4b2f-9f3f-886db4604e01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:10.503336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4922be-6c28-4b2f-9f3f-886db4604e01 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:10.525906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-bb4922be-6c28-4b2f-9f3f-886db4604e01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:10.525906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:10.525906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:10.525906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:10.525906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:10.525906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:10.525906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:10.525906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:10.525906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:10.525906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:10.525906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:10.525906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:10.525906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:10.525906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:10.525906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:10.526875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:10.527772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:10.527772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:10.527772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:10.527772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:10.530744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 434/850] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:10.536248 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac013dc-b256-432a-a55a-5ac9bd184137 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:10.537048 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4922be-6c28-4b2f-9f3f-886db4604e01 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:10.538007 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac013dc-b256-432a-a55a-5ac9bd184137 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:10.582402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:10.628686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:10.654545 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4922be-6c28-4b2f-9f3f-886db4604e01 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:10.665733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 417/851] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:31:10 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 10:31:10.667060 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1c949-56bc-4888-8fda-d00b9011e2cc None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:10.667349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1c949-56bc-4888-8fda-d00b9011e2cc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:10.667623 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1c949-56bc-4888-8fda-d00b9011e2cc None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:10.673075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 418/852] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:10.673075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1c949-56bc-4888-8fda-d00b9011e2cc None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:10.705509 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01ce7a-2c39-444b-a3f1-607a17361402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:10.706314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01ce7a-2c39-444b-a3f1-607a17361402 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:10.706671 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01ce7a-2c39-444b-a3f1-607a17361402 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:10.707026 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01ce7a-2c39-444b-a3f1-607a17361402 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:10.707856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01ce7a-2c39-444b-a3f1-607a17361402 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:10.708239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01ce7a-2c39-444b-a3f1-607a17361402 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:10.709700 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01ce7a-2c39-444b-a3f1-607a17361402 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:10.723333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 435/853] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:10.724627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefab08-6d80-482e-a99d-edd625f6e9f8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:10.724923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefab08-6d80-482e-a99d-edd625f6e9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:10.725199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefab08-6d80-482e-a99d-edd625f6e9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:10.744471 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-dcefab08-6d80-482e-a99d-edd625f6e9f8 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:10.746107 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-dcefab08-6d80-482e-a99d-edd625f6e9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:10.797172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1c949-56bc-4888-8fda-d00b9011e2cc None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:10.823243 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 436/854] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:10.824208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefab08-6d80-482e-a99d-edd625f6e9f8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:10.845331 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f9faf-118d-43c6-8c6f-7196db40406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:10.847163 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f9faf-118d-43c6-8c6f-7196db40406f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:10.847690 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f9faf-118d-43c6-8c6f-7196db40406f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:10.848115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f9faf-118d-43c6-8c6f-7196db40406f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:10.856414 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:10.861767 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f9faf-118d-43c6-8c6f-7196db40406f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:10.863668 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f9faf-118d-43c6-8c6f-7196db40406f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:10.882632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-f59f9faf-118d-43c6-8c6f-7196db40406f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:10.882632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:10.882632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:10.882632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:10.882632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:10.882632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:10.882632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:10.882632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:10.882632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:10.882632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:10.882632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:10.882632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:10.882632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:10.882632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:10.882632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:10.883578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:10.884562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:10.884562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:10.884562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:10.884562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:10.884562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 419/855] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:10.885341 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f9faf-118d-43c6-8c6f-7196db40406f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:10.914977 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:10.952146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefab08-6d80-482e-a99d-edd625f6e9f8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:10.987831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f14e7c-27fd-46a9-a7ab-9da3b5ac3dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:10.988469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f14e7c-27fd-46a9-a7ab-9da3b5ac3dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:10.988747 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f14e7c-27fd-46a9-a7ab-9da3b5ac3dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:10.989014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f14e7c-27fd-46a9-a7ab-9da3b5ac3dc5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:10.993061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f9faf-118d-43c6-8c6f-7196db40406f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:10.996064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f14e7c-27fd-46a9-a7ab-9da3b5ac3dc5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:10.997502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f14e7c-27fd-46a9-a7ab-9da3b5ac3dc5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-31f14e7c-27fd-46a9-a7ab-9da3b5ac3dc5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:31:11.009350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:11.010519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:11.011762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:11.011762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:11.011762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:31:11.011762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:31:11.011762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:31:11.011762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:31:11.011762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:31:11.011762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:31:11.011762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 10:31:11.011762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:11.011762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 437/856] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:31:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:11.011762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f14e7c-27fd-46a9-a7ab-9da3b5ac3dc5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:11.038117 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45013f2-7848-4afd-a269-5052082b54b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:11.038971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45013f2-7848-4afd-a269-5052082b54b6 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:11.039294 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45013f2-7848-4afd-a269-5052082b54b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:11.039573 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45013f2-7848-4afd-a269-5052082b54b6 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:11.040279 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45013f2-7848-4afd-a269-5052082b54b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:11.041681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45013f2-7848-4afd-a269-5052082b54b6 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:11.056828 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 420/857] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 3 10:31:10 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:31:11.058145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a165ea8-3428-4a41-8d69-0de8e40447e1 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:11.058430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a165ea8-3428-4a41-8d69-0de8e40447e1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:11.058700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a165ea8-3428-4a41-8d69-0de8e40447e1 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:11.059965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 421/858] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:11.061762 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a165ea8-3428-4a41-8d69-0de8e40447e1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:11.112408 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:11.134076 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f14e7c-27fd-46a9-a7ab-9da3b5ac3dc5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:11.167940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:11.181670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96f0a1-f125-4ef8-9968-5f11c6c52ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:11.182312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96f0a1-f125-4ef8-9968-5f11c6c52ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:11.182595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96f0a1-f125-4ef8-9968-5f11c6c52ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:11.182881 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96f0a1-f125-4ef8-9968-5f11c6c52ca6 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:11.183580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96f0a1-f125-4ef8-9968-5f11c6c52ca6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:11.183947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96f0a1-f125-4ef8-9968-5f11c6c52ca6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:11.184443 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a165ea8-3428-4a41-8d69-0de8e40447e1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:11.185329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96f0a1-f125-4ef8-9968-5f11c6c52ca6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:11.201363 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 438/859] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:11.202730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29290c1-6144-490a-9026-3c366a8b620e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:11.203034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29290c1-6144-490a-9026-3c366a8b620e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:11.203296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29290c1-6144-490a-9026-3c366a8b620e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:11.227041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f29290c1-6144-490a-9026-3c366a8b620e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:11.229011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f29290c1-6144-490a-9026-3c366a8b620e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:11.232470 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810af94e-719b-4dd0-bd75-b11179c08176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:11.232628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810af94e-719b-4dd0-bd75-b11179c08176 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:11.232969 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810af94e-719b-4dd0-bd75-b11179c08176 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:11.233159 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810af94e-719b-4dd0-bd75-b11179c08176 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:11.233978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810af94e-719b-4dd0-bd75-b11179c08176 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:11.234287 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810af94e-719b-4dd0-bd75-b11179c08176 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:11.235709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810af94e-719b-4dd0-bd75-b11179c08176 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:11.258194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 422/860] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Aug 3 10:31:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:11.258961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810af94e-719b-4dd0-bd75-b11179c08176 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:11.307497 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 439/861] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:11.308844 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29290c1-6144-490a-9026-3c366a8b620e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:11.375549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810af94e-719b-4dd0-bd75-b11179c08176 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:11.395835 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:11.413063 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2a1b3-c656-4a71-b7d2-f9f509c0aa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:11.413752 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2a1b3-c656-4a71-b7d2-f9f509c0aa87 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:11.414035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2a1b3-c656-4a71-b7d2-f9f509c0aa87 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:11.414307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2a1b3-c656-4a71-b7d2-f9f509c0aa87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:11.421069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2a1b3-c656-4a71-b7d2-f9f509c0aa87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:11.422536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2a1b3-c656-4a71-b7d2-f9f509c0aa87 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:11.429156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29290c1-6144-490a-9026-3c366a8b620e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:11.434288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-cee2a1b3-c656-4a71-b7d2-f9f509c0aa87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:11.434288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:11.434288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:11.434288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:11.434288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:11.434288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:11.434288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:11.434288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:11.434288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:11.434288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:11.434288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:11.434288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:11.434288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:31:11.434288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:31:11.434288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:11.435174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:11.436085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:31:11.436085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:31:11.436085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:31:11.436085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:31:11.436085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:31:11.436085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:31:11.436085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:11.436085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:11.440908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 423/862] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:11.441591 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2a1b3-c656-4a71-b7d2-f9f509c0aa87 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:11.450612 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:11.467995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dac72e-62d0-4d18-923c-25f2ba15c5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:11.468746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dac72e-62d0-4d18-923c-25f2ba15c5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:11.469134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dac72e-62d0-4d18-923c-25f2ba15c5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:11.469549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dac72e-62d0-4d18-923c-25f2ba15c5a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:11.545869 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dac72e-62d0-4d18-923c-25f2ba15c5a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:11.546709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2a1b3-c656-4a71-b7d2-f9f509c0aa87 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:11.547323 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dac72e-62d0-4d18-923c-25f2ba15c5a0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:11.583644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542fe58-d2e1-44b2-9b8d-08244a9fc05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:11.584499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542fe58-d2e1-44b2-9b8d-08244a9fc05a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:11.584877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542fe58-d2e1-44b2-9b8d-08244a9fc05a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:11.585250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542fe58-d2e1-44b2-9b8d-08244a9fc05a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:11.586038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542fe58-d2e1-44b2-9b8d-08244a9fc05a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:11.587595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542fe58-d2e1-44b2-9b8d-08244a9fc05a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-b542fe58-d2e1-44b2-9b8d-08244a9fc05a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:11.598630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:11.599782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:11.600811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:11.600811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:11.600811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:11.600811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:11.600811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:11.600811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:11.600811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:11.600811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:11.600811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:11.600811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:11.600811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:11.600811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:11.600811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:11.600811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 10:31:11.600811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:11.605321 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 424/863] 10.222.0.47 () {56 vars in 1132 bytes} [Wed Aug 3 10:31:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:11.606834 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542fe58-d2e1-44b2-9b8d-08244a9fc05a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:11.622458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:11.662252 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 440/864] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:31:11 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 10:31:11.663091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dac72e-62d0-4d18-923c-25f2ba15c5a0 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:11.693235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:11.729480 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542fe58-d2e1-44b2-9b8d-08244a9fc05a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:11.767280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd083e23-9cb2-4a21-8837-8d4befb8fc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:11.767959 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd083e23-9cb2-4a21-8837-8d4befb8fc82 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:11.768243 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd083e23-9cb2-4a21-8837-8d4befb8fc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:11.768521 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd083e23-9cb2-4a21-8837-8d4befb8fc82 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:11.769283 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd083e23-9cb2-4a21-8837-8d4befb8fc82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:11.769622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd083e23-9cb2-4a21-8837-8d4befb8fc82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:11.771079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd083e23-9cb2-4a21-8837-8d4befb8fc82 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:11.778437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dac72e-62d0-4d18-923c-25f2ba15c5a0 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:11.784078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 425/865] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:31:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:11.785244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd083e23-9cb2-4a21-8837-8d4befb8fc82 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:11.818043 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06168a87-0387-4ec9-87c1-42e81bc3656c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:11.818673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06168a87-0387-4ec9-87c1-42e81bc3656c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:11.818954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06168a87-0387-4ec9-87c1-42e81bc3656c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:11.819307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06168a87-0387-4ec9-87c1-42e81bc3656c None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:11.819913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06168a87-0387-4ec9-87c1-42e81bc3656c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:11.820221 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06168a87-0387-4ec9-87c1-42e81bc3656c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:11.821613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06168a87-0387-4ec9-87c1-42e81bc3656c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:11.826744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 441/866] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 3 10:31:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:11.830417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06168a87-0387-4ec9-87c1-42e81bc3656c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:11.868903 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:11.894484 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd083e23-9cb2-4a21-8837-8d4befb8fc82 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:11.923340 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:11.931666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba4541-0725-4023-ad7a-15db3b936e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:11.932401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba4541-0725-4023-ad7a-15db3b936e2b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:11.932764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba4541-0725-4023-ad7a-15db3b936e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:11.933048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba4541-0725-4023-ad7a-15db3b936e2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:11.944876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba4541-0725-4023-ad7a-15db3b936e2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:11.946381 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba4541-0725-4023-ad7a-15db3b936e2b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:11.947254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06168a87-0387-4ec9-87c1-42e81bc3656c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:11.961950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-85ba4541-0725-4023-ad7a-15db3b936e2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:11.961950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:11.961950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:11.961950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:11.961950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:11.961950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:11.961950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:11.961950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:11.961950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:11.961950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:11.961950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:11.961950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:11.961950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:11.961950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:11.961950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:11.962852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:11.963730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:11.963730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:11.963730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:11.963730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:11.963730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 426/867] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:11.964003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba4541-0725-4023-ad7a-15db3b936e2b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:11.988683 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100d0ff-35f8-4ca0-a17a-bc69ce9acc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:11.989410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100d0ff-35f8-4ca0-a17a-bc69ce9acc94 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:11.989786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100d0ff-35f8-4ca0-a17a-bc69ce9acc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:11.990058 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100d0ff-35f8-4ca0-a17a-bc69ce9acc94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:11.990969 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100d0ff-35f8-4ca0-a17a-bc69ce9acc94 None admin] RBAC: Adding query filter params (role.id=1df33ad923a44a59b50dcacb6a75fefb, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=035ae78ef3cb43e985e06cc3fa971c8b) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:11.991273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100d0ff-35f8-4ca0-a17a-bc69ce9acc94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:11.992791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100d0ff-35f8-4ca0-a17a-bc69ce9acc94 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:12.009495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 442/868] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:31:11 2022] GET /identity/v3/role_assignments?user.id=035ae78ef3cb43e985e06cc3fa971c8b&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=1df33ad923a44a59b50dcacb6a75fefb => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:12.009495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100d0ff-35f8-4ca0-a17a-bc69ce9acc94 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:12.089746 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba4541-0725-4023-ad7a-15db3b936e2b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:12.138917 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180644a-bd45-4729-b718-c175fdce862c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:12.139853 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180644a-bd45-4729-b718-c175fdce862c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:12.140207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180644a-bd45-4729-b718-c175fdce862c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:12.140491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180644a-bd45-4729-b718-c175fdce862c None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:12.141619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180644a-bd45-4729-b718-c175fdce862c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:12.142924 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180644a-bd45-4729-b718-c175fdce862c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:12.144658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180644a-bd45-4729-b718-c175fdce862c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:12.145326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:12.149782 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100d0ff-35f8-4ca0-a17a-bc69ce9acc94 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:12.151977 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 427/869] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 3 10:31:11 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:12.198234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042eeb5-3724-4dd5-acb6-cd0bd912af26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:12.198842 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042eeb5-3724-4dd5-acb6-cd0bd912af26 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:12.199162 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042eeb5-3724-4dd5-acb6-cd0bd912af26 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:12.199451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042eeb5-3724-4dd5-acb6-cd0bd912af26 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:12.200230 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e042eeb5-3724-4dd5-acb6-cd0bd912af26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:12.200543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e042eeb5-3724-4dd5-acb6-cd0bd912af26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:12.202043 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e042eeb5-3724-4dd5-acb6-cd0bd912af26 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:12.217630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 443/870] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:12.228391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180644a-bd45-4729-b718-c175fdce862c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:12.290059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:12.352306 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180644a-bd45-4729-b718-c175fdce862c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:12.389063 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b8738-e9e7-4ea3-bc42-a6ddc574d67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:12.389810 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b8738-e9e7-4ea3-bc42-a6ddc574d67e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:12.390093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b8738-e9e7-4ea3-bc42-a6ddc574d67e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:12.390370 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b8738-e9e7-4ea3-bc42-a6ddc574d67e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:12.401881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b8738-e9e7-4ea3-bc42-a6ddc574d67e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:12.403576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b8738-e9e7-4ea3-bc42-a6ddc574d67e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:12.416307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-297b8738-e9e7-4ea3-bc42-a6ddc574d67e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:12.416307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:12.416307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:12.416307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:12.416307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:12.416307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:12.416307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:12.416307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:12.416307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:12.416307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:12.416307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:12.416307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:12.416307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:31:12.416307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:31:12.416307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:12.417206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:12.418130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:31:12.418130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:31:12.418130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:31:12.418130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:31:12.418130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:31:12.418130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:31:12.418130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:12.418130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:12.420258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 428/871] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:12.426633 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042eeb5-3724-4dd5-acb6-cd0bd912af26 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:12.534513 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042eeb5-3724-4dd5-acb6-cd0bd912af26 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:12.575009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab5344-cced-4b78-b4e4-c68e5cb6bc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:12.575646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab5344-cced-4b78-b4e4-c68e5cb6bc3c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:12.575994 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab5344-cced-4b78-b4e4-c68e5cb6bc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:12.576281 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab5344-cced-4b78-b4e4-c68e5cb6bc3c None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:12.577046 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab5344-cced-4b78-b4e4-c68e5cb6bc3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:12.577357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab5344-cced-4b78-b4e4-c68e5cb6bc3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:12.578809 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab5344-cced-4b78-b4e4-c68e5cb6bc3c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:12.592959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 444/872] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:31:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:12.599210 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b8738-e9e7-4ea3-bc42-a6ddc574d67e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:12.710368 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b8738-e9e7-4ea3-bc42-a6ddc574d67e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:12.718160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:12.747165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb317f-8064-468a-94c0-07921d659027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:12.747795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb317f-8064-468a-94c0-07921d659027 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:12.748085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb317f-8064-468a-94c0-07921d659027 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:12.748360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb317f-8064-468a-94c0-07921d659027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:12.749158 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb317f-8064-468a-94c0-07921d659027 None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=a81ea5afe74b43879abc94b3cd134fd2) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:12.749492 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb317f-8064-468a-94c0-07921d659027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:12.750922 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb317f-8064-468a-94c0-07921d659027 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:12.757671 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 429/873] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:31:12 2022] GET /identity/v3/role_assignments?user.id=a81ea5afe74b43879abc94b3cd134fd2&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=01714797cab143e1a7a150c068fa9315 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:13.164279 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287d19d-bad0-4700-b86d-4e1f2549a206 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:13.164909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287d19d-bad0-4700-b86d-4e1f2549a206 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:13.164909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287d19d-bad0-4700-b86d-4e1f2549a206 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:13.166462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 445/874] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:13.171840 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529554c-7682-43e7-ab03-09fdd657910a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:13.172242 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529554c-7682-43e7-ab03-09fdd657910a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:13.172444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529554c-7682-43e7-ab03-09fdd657910a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:13.191360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7529554c-7682-43e7-ab03-09fdd657910a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:13.192577 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7529554c-7682-43e7-ab03-09fdd657910a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:13.268816 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 430/875] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:13 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:13.290931 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287d19d-bad0-4700-b86d-4e1f2549a206 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:13.311502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:13.407341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287d19d-bad0-4700-b86d-4e1f2549a206 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:13.446257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cce9e-25c3-42f2-a399-31f38e6b0455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:13.446933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cce9e-25c3-42f2-a399-31f38e6b0455 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:13.447213 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cce9e-25c3-42f2-a399-31f38e6b0455 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:13.447486 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cce9e-25c3-42f2-a399-31f38e6b0455 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:13.448124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cce9e-25c3-42f2-a399-31f38e6b0455 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:13.449510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cce9e-25c3-42f2-a399-31f38e6b0455 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-bf6cce9e-25c3-42f2-a399-31f38e6b0455 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:13.459867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:13.460933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:13.462012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:13.462012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:13.462012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:13.462012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:13.462012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:13.462012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:13.462012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:13.462012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:13.462012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:13.462012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:13.462012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:13.462012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:13.462012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 10:31:13.462012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:13.462012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 446/876] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:31:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:13.468434 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529554c-7682-43e7-ab03-09fdd657910a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:13.577517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:13.580595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529554c-7682-43e7-ab03-09fdd657910a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:13.620427 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01370a85-b0b0-4f87-9523-65aba6bbdd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:13.621063 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01370a85-b0b0-4f87-9523-65aba6bbdd91 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:13.621350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01370a85-b0b0-4f87-9523-65aba6bbdd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:13.621659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01370a85-b0b0-4f87-9523-65aba6bbdd91 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:13.622348 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01370a85-b0b0-4f87-9523-65aba6bbdd91 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:13.622653 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01370a85-b0b0-4f87-9523-65aba6bbdd91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:13.624042 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01370a85-b0b0-4f87-9523-65aba6bbdd91 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:13.630281 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 431/877] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:31:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:13.635940 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cce9e-25c3-42f2-a399-31f38e6b0455 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:13.729137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:13.756002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cce9e-25c3-42f2-a399-31f38e6b0455 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:13.791932 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce9501-3a90-4ac0-a5cb-4e69797a5359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:13.792621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce9501-3a90-4ac0-a5cb-4e69797a5359 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:13.792928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce9501-3a90-4ac0-a5cb-4e69797a5359 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:13.793206 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce9501-3a90-4ac0-a5cb-4e69797a5359 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:13.793939 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce9501-3a90-4ac0-a5cb-4e69797a5359 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:13.794246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce9501-3a90-4ac0-a5cb-4e69797a5359 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:13.795606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce9501-3a90-4ac0-a5cb-4e69797a5359 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:13.801515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 447/878] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:31:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:13.807399 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01370a85-b0b0-4f87-9523-65aba6bbdd91 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:13.914717 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01370a85-b0b0-4f87-9523-65aba6bbdd91 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:13.950088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d801347-7545-4e15-8dbc-bbebe292a1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:13.950711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d801347-7545-4e15-8dbc-bbebe292a1bd None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:13.950990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d801347-7545-4e15-8dbc-bbebe292a1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:13.951262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d801347-7545-4e15-8dbc-bbebe292a1bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:13.952020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d801347-7545-4e15-8dbc-bbebe292a1bd None admin] RBAC: Adding query filter params (interface=public, service_id=dc77b4897611436f8a744fd2ca4b2269, region_id=RegionOne) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:13.952396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d801347-7545-4e15-8dbc-bbebe292a1bd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:13.953835 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d801347-7545-4e15-8dbc-bbebe292a1bd None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:13.959867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 432/879] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 3 10:31:13 2022] GET /identity/v3/endpoints?service_id=dc77b4897611436f8a744fd2ca4b2269&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:13.965210 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce9501-3a90-4ac0-a5cb-4e69797a5359 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:14.074482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce9501-3a90-4ac0-a5cb-4e69797a5359 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:14.113128 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2546055-a00d-4c59-b529-151acaacb568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:14.113787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2546055-a00d-4c59-b529-151acaacb568 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:14.114065 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2546055-a00d-4c59-b529-151acaacb568 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:14.114337 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2546055-a00d-4c59-b529-151acaacb568 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:14.114998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2546055-a00d-4c59-b529-151acaacb568 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:14.116351 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2546055-a00d-4c59-b529-151acaacb568 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:14.122605 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 448/880] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 3 10:31:13 2022] GET /identity/v3/services => generated 1011 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:31:14.369306 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f61a7-9a01-478c-8af9-86794213085b None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:14.369832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f61a7-9a01-478c-8af9-86794213085b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:14.370061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f61a7-9a01-478c-8af9-86794213085b None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:14.371859 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 433/881] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:14.377318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a75cb-0538-4bb3-b195-5ed3cb3be347 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:14.377797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a75cb-0538-4bb3-b195-5ed3cb3be347 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:14.377874 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a75cb-0538-4bb3-b195-5ed3cb3be347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:14.397114 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f86a75cb-0538-4bb3-b195-5ed3cb3be347 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:14.398346 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f86a75cb-0538-4bb3-b195-5ed3cb3be347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:14.401247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:14.473632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 449/882] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:14 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:14.479108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cba79d-da54-4d17-8bbb-7a652d98c081 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:14.479758 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cba79d-da54-4d17-8bbb-7a652d98c081 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:14.480380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cba79d-da54-4d17-8bbb-7a652d98c081 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:14.482279 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 434/883] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:14.487705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fda1a-da91-4b6f-96c6-d0e0a745ad96 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:14.487999 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fda1a-da91-4b6f-96c6-d0e0a745ad96 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:14.488275 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fda1a-da91-4b6f-96c6-d0e0a745ad96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:14.496780 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cba79d-da54-4d17-8bbb-7a652d98c081 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:14.508717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d54fda1a-da91-4b6f-96c6-d0e0a745ad96 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:14.510272 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d54fda1a-da91-4b6f-96c6-d0e0a745ad96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:14.529852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:14.601563 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 450/884] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:14 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:14.614419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cba79d-da54-4d17-8bbb-7a652d98c081 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:14.621202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fda1a-da91-4b6f-96c6-d0e0a745ad96 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:14.668494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c162a1-3373-46ab-bfe9-8b1c7d2d0b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:14.669571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c162a1-3373-46ab-bfe9-8b1c7d2d0b4c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:14.670063 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c162a1-3373-46ab-bfe9-8b1c7d2d0b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:14.670623 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c162a1-3373-46ab-bfe9-8b1c7d2d0b4c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:14.686804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c162a1-3373-46ab-bfe9-8b1c7d2d0b4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-d2c162a1-3373-46ab-bfe9-8b1c7d2d0b4c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:31:14.695809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:14.696793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:14.697670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:14.697670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:14.697670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:14.697670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:14.697670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:31:14.697670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:31:14.697670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:31:14.697670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:31:14.697670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:31:14.697670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:31:14.697670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 10:31:14.697670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:14.699264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 435/885] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Aug 3 10:31:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:14.706442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c162a1-3373-46ab-bfe9-8b1c7d2d0b4c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:14.721124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:14.756948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fda1a-da91-4b6f-96c6-d0e0a745ad96 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:14.797768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7c5a3-3f11-4356-8ba9-dc3d552bc8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:14.798405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7c5a3-3f11-4356-8ba9-dc3d552bc8ff None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:14.798798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7c5a3-3f11-4356-8ba9-dc3d552bc8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:14.798995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7c5a3-3f11-4356-8ba9-dc3d552bc8ff None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:14.799710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7c5a3-3f11-4356-8ba9-dc3d552bc8ff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:14.801182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7c5a3-3f11-4356-8ba9-dc3d552bc8ff None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:14.819934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 451/886] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 3 10:31:14 2022] POST /identity/v3/services => generated 269 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:31:14.828253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:14.839733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c162a1-3373-46ab-bfe9-8b1c7d2d0b4c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:14.887022 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c9207-376e-4d3f-9ac7-a3fec157acd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:14.888038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c9207-376e-4d3f-9ac7-a3fec157acd7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:14.888539 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c9207-376e-4d3f-9ac7-a3fec157acd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:14.889022 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c9207-376e-4d3f-9ac7-a3fec157acd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:14.890000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c9207-376e-4d3f-9ac7-a3fec157acd7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:14.890550 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c9207-376e-4d3f-9ac7-a3fec157acd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:14.892164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c9207-376e-4d3f-9ac7-a3fec157acd7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:14.903011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 436/887] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Aug 3 10:31:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:14.909570 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7c5a3-3f11-4356-8ba9-dc3d552bc8ff None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:14.970350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba87cb-f276-4f46-83c8-db7563f8764f None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:14.970678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba87cb-f276-4f46-83c8-db7563f8764f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:14.970937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba87cb-f276-4f46-83c8-db7563f8764f None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:14.973336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 437/888] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:31:14.979229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1093c3-7297-4eef-9774-682d6e28e4eb None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:14.979528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1093c3-7297-4eef-9774-682d6e28e4eb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:14.979788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1093c3-7297-4eef-9774-682d6e28e4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:15.001149 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-cd1093c3-7297-4eef-9774-682d6e28e4eb None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:15.003055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cd1093c3-7297-4eef-9774-682d6e28e4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:15.041854 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7c5a3-3f11-4356-8ba9-dc3d552bc8ff None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:15.084915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:15.097808 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f7884-06c1-43e5-b846-ce0e71f6dc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:15.098666 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f7884-06c1-43e5-b846-ce0e71f6dc76 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:15.098929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f7884-06c1-43e5-b846-ce0e71f6dc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:15.099175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f7884-06c1-43e5-b846-ce0e71f6dc76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:15.100280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 438/889] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:15.111859 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f7884-06c1-43e5-b846-ce0e71f6dc76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:15.113302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f7884-06c1-43e5-b846-ce0e71f6dc76 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:15.131580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-f73f7884-06c1-43e5-b846-ce0e71f6dc76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:15.131580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:15.131580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:15.131580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:15.131580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:15.131580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:15.131580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:15.131580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:15.131580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:15.131580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:15.131580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:15.131580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:15.131580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:15.131580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:15.131580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:15.132524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:15.133391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:15.133391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:15.133391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:15.133391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:15.133391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 452/890] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:15.135007 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1093c3-7297-4eef-9774-682d6e28e4eb None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:15.139466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f7884-06c1-43e5-b846-ce0e71f6dc76 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:15.246723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:15.281439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1093c3-7297-4eef-9774-682d6e28e4eb None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:15.281834 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f7884-06c1-43e5-b846-ce0e71f6dc76 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:15.332205 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf30da-b96c-4de6-97b9-bb62f490f10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:15.332878 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf30da-b96c-4de6-97b9-bb62f490f10e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:15.333655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf30da-b96c-4de6-97b9-bb62f490f10e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:15.333927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf30da-b96c-4de6-97b9-bb62f490f10e None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:15.334990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf30da-b96c-4de6-97b9-bb62f490f10e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:15.335304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf30da-b96c-4de6-97b9-bb62f490f10e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:15.336877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf30da-b96c-4de6-97b9-bb62f490f10e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:15.340666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cd60e-9d85-4707-b774-8ba68705154f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:15.341707 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cd60e-9d85-4707-b774-8ba68705154f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:15.342216 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cd60e-9d85-4707-b774-8ba68705154f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:15.342694 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cd60e-9d85-4707-b774-8ba68705154f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:15.343537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6cd60e-9d85-4707-b774-8ba68705154f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:15.345198 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6cd60e-9d85-4707-b774-8ba68705154f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:15.355347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 453/891] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-fb6cd60e-9d85-4707-b774-8ba68705154f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:15.358600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:15.359587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:15.360616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:15.360616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:15.360616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:15.360616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:15.360616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:15.360616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:15.360616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:15.360616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:15.360616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:15.360616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:15.360616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:15.360616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:15.360616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 10:31:15.360616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:15.362556 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf30da-b96c-4de6-97b9-bb62f490f10e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:15.364376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 439/892] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 3 10:31:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:15.370770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cd60e-9d85-4707-b774-8ba68705154f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:15.415835 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:15.484323 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf30da-b96c-4de6-97b9-bb62f490f10e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:15.516190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cd60e-9d85-4707-b774-8ba68705154f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:15.548460 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc2358-d55b-4c8e-82da-bdb8b168c501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:15.549133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc2358-d55b-4c8e-82da-bdb8b168c501 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:15.549740 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc2358-d55b-4c8e-82da-bdb8b168c501 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:15.550028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc2358-d55b-4c8e-82da-bdb8b168c501 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:15.562725 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc2358-d55b-4c8e-82da-bdb8b168c501 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:15.564694 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc2358-d55b-4c8e-82da-bdb8b168c501 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:15.566686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bfe11-378b-4037-a107-a40e686e81f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:15.567400 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bfe11-378b-4037-a107-a40e686e81f6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:15.567842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bfe11-378b-4037-a107-a40e686e81f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:15.568190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bfe11-378b-4037-a107-a40e686e81f6 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:15.568954 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bfe11-378b-4037-a107-a40e686e81f6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:15.569358 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bfe11-378b-4037-a107-a40e686e81f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:15.570836 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bfe11-378b-4037-a107-a40e686e81f6 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:15.576479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 440/893] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 3 10:31:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-7afc2358-d55b-4c8e-82da-bdb8b168c501 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:31:15.578799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:15.579722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:15.581228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:15.581228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:15.581228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:31:15.581228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:31:15.581228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:31:15.581228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:31:15.581228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:31:15.581228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:31:15.581228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 10:31:15.581228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:15.581228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 454/894] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 3 10:31:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:15.585908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bfe11-378b-4037-a107-a40e686e81f6 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:15.587920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc2358-d55b-4c8e-82da-bdb8b168c501 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:15.598654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:15.702812 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc2358-d55b-4c8e-82da-bdb8b168c501 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:15.704663 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:15.724148 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bfe11-378b-4037-a107-a40e686e81f6 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:15.747279 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd18440-fbad-4b20-904b-351ddf1215c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:15.748651 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd18440-fbad-4b20-904b-351ddf1215c7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:15.749150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd18440-fbad-4b20-904b-351ddf1215c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:15.749650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd18440-fbad-4b20-904b-351ddf1215c7 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:15.750653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd18440-fbad-4b20-904b-351ddf1215c7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:15.751145 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd18440-fbad-4b20-904b-351ddf1215c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:15.754296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd18440-fbad-4b20-904b-351ddf1215c7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:15.776840 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dd296-269e-4659-be8e-b5755bfea190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:15.778047 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dd296-269e-4659-be8e-b5755bfea190 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:15.778454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dd296-269e-4659-be8e-b5755bfea190 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:15.778801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dd296-269e-4659-be8e-b5755bfea190 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:15.779572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769dd296-269e-4659-be8e-b5755bfea190 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:15.779955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769dd296-269e-4659-be8e-b5755bfea190 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:15.780863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 455/895] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Aug 3 10:31:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:15.782289 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769dd296-269e-4659-be8e-b5755bfea190 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:15.786498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd18440-fbad-4b20-904b-351ddf1215c7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:15.797521 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 441/896] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 3 10:31:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:15.918103 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd18440-fbad-4b20-904b-351ddf1215c7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:15.959969 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a9e7d-9485-4697-8486-e8a53247211a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:15.960557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a9e7d-9485-4697-8486-e8a53247211a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:15.960845 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a9e7d-9485-4697-8486-e8a53247211a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:15.961125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a9e7d-9485-4697-8486-e8a53247211a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:15.992306 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:16.035138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a9e7d-9485-4697-8486-e8a53247211a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:16.036598 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a9e7d-9485-4697-8486-e8a53247211a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:16.141942 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 456/897] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:31:15 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 10:31:16.147694 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dd296-269e-4659-be8e-b5755bfea190 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:16.239654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:16.280316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dd296-269e-4659-be8e-b5755bfea190 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:16.300838 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1cad2-1ec1-4156-aa5b-b48a870f2791 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:16.301231 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1cad2-1ec1-4156-aa5b-b48a870f2791 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:16.301814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1cad2-1ec1-4156-aa5b-b48a870f2791 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:16.303203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 457/898] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:16.308293 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b41ffa-6b58-442b-a5f6-e7c266b0b6e6 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:16.308584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b41ffa-6b58-442b-a5f6-e7c266b0b6e6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:16.308872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b41ffa-6b58-442b-a5f6-e7c266b0b6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:16.328119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-30b41ffa-6b58-442b-a5f6-e7c266b0b6e6 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:16.328527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be78638-a99e-4acd-872d-c7487e0a6385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:16.329189 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be78638-a99e-4acd-872d-c7487e0a6385 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:16.329362 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-30b41ffa-6b58-442b-a5f6-e7c266b0b6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:16.329550 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be78638-a99e-4acd-872d-c7487e0a6385 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:16.329784 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be78638-a99e-4acd-872d-c7487e0a6385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:16.349213 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be78638-a99e-4acd-872d-c7487e0a6385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:16.351018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be78638-a99e-4acd-872d-c7487e0a6385 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:16.370124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-2be78638-a99e-4acd-872d-c7487e0a6385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:16.370124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:16.370124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:16.370124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:16.370124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:16.370124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:16.370124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:16.370124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:16.370124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:16.370124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:16.370124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:16.370124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:16.370124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:16.370124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:16.370124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:16.371493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:16.372407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:16.372407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:16.372407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:16.372407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:16.375667 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 442/899] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:16.381749 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be78638-a99e-4acd-872d-c7487e0a6385 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:16.403000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:16.421605 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 458/900] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:16.444497 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b41ffa-6b58-442b-a5f6-e7c266b0b6e6 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:16.511430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be78638-a99e-4acd-872d-c7487e0a6385 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:16.543182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:16.564866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df625e-da15-44f6-94ed-1a61563d178d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:16.566594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df625e-da15-44f6-94ed-1a61563d178d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:16.567180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df625e-da15-44f6-94ed-1a61563d178d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:16.567900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df625e-da15-44f6-94ed-1a61563d178d None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:16.569804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df625e-da15-44f6-94ed-1a61563d178d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:16.570960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df625e-da15-44f6-94ed-1a61563d178d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:16.574227 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df625e-da15-44f6-94ed-1a61563d178d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:16.578913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b41ffa-6b58-442b-a5f6-e7c266b0b6e6 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:16.612821 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 443/901] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:16.616675 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df625e-da15-44f6-94ed-1a61563d178d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:16.626245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef385587-30c5-4090-a08e-9f1ea901c00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:16.626943 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef385587-30c5-4090-a08e-9f1ea901c00e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:16.627228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef385587-30c5-4090-a08e-9f1ea901c00e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:16.627507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef385587-30c5-4090-a08e-9f1ea901c00e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:16.628200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef385587-30c5-4090-a08e-9f1ea901c00e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:16.629662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef385587-30c5-4090-a08e-9f1ea901c00e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ef385587-30c5-4090-a08e-9f1ea901c00e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:16.639557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:16.641535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:16.644202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:16.644202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:16.644202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:16.644202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:16.644202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:16.644202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:16.644202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:16.644202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:16.644202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:16.644202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:16.644202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:16.644202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:16.644202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 10:31:16.644202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:16.644202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 459/902] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:31:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:16.647720 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef385587-30c5-4090-a08e-9f1ea901c00e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:16.739127 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:16.779454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df625e-da15-44f6-94ed-1a61563d178d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:16.793711 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef385587-30c5-4090-a08e-9f1ea901c00e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:16.826763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391afae7-68b0-493c-a6d6-395750ca5cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:16.827736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391afae7-68b0-493c-a6d6-395750ca5cdb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:16.828141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391afae7-68b0-493c-a6d6-395750ca5cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:16.828594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391afae7-68b0-493c-a6d6-395750ca5cdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:16.836562 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69621ee1-7ab1-48d2-8f71-2d86acffc594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:16.837210 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391afae7-68b0-493c-a6d6-395750ca5cdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:16.837619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621ee1-7ab1-48d2-8f71-2d86acffc594 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:16.838056 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621ee1-7ab1-48d2-8f71-2d86acffc594 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:16.838487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621ee1-7ab1-48d2-8f71-2d86acffc594 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:16.839103 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391afae7-68b0-493c-a6d6-395750ca5cdb None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:16.839741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69621ee1-7ab1-48d2-8f71-2d86acffc594 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:16.840342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69621ee1-7ab1-48d2-8f71-2d86acffc594 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:16.842563 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69621ee1-7ab1-48d2-8f71-2d86acffc594 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:16.848415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 460/903] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:31:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:16.853852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69621ee1-7ab1-48d2-8f71-2d86acffc594 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:16.857077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-391afae7-68b0-493c-a6d6-395750ca5cdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:16.857077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:16.857077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:16.857077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:16.857077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:16.857077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:16.857077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:16.857077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:16.857077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:16.857077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:16.857077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:16.857077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:16.857077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:31:16.857077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:31:16.857077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:16.858196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:16.859142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:31:16.859142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:31:16.859142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:31:16.859142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:31:16.859142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:31:16.859142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:31:16.859142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:16.859142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:16.863251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 444/904] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:16.864230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391afae7-68b0-493c-a6d6-395750ca5cdb None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:16.916764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:16.989718 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391afae7-68b0-493c-a6d6-395750ca5cdb None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:17.003886 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69621ee1-7ab1-48d2-8f71-2d86acffc594 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:17.026621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:17.043407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1242eb5-8135-4b60-bfa8-99157cefa65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:17.044077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1242eb5-8135-4b60-bfa8-99157cefa65f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:17.044410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1242eb5-8135-4b60-bfa8-99157cefa65f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:17.044808 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1242eb5-8135-4b60-bfa8-99157cefa65f None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:17.045896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1242eb5-8135-4b60-bfa8-99157cefa65f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:17.046280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1242eb5-8135-4b60-bfa8-99157cefa65f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:17.047308 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45b5ef-aed0-4393-88cb-8156b734745e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:17.047892 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1242eb5-8135-4b60-bfa8-99157cefa65f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:17.048260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45b5ef-aed0-4393-88cb-8156b734745e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:17.048715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45b5ef-aed0-4393-88cb-8156b734745e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:17.049142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45b5ef-aed0-4393-88cb-8156b734745e None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:17.050357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45b5ef-aed0-4393-88cb-8156b734745e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:17.050819 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45b5ef-aed0-4393-88cb-8156b734745e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:17.052897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45b5ef-aed0-4393-88cb-8156b734745e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:17.060241 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 461/905] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 3 10:31:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:17.065506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 445/906] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:31:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:17.074650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45b5ef-aed0-4393-88cb-8156b734745e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:17.076551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1242eb5-8135-4b60-bfa8-99157cefa65f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:17.210177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45b5ef-aed0-4393-88cb-8156b734745e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:17.215794 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:17.219203 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1242eb5-8135-4b60-bfa8-99157cefa65f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:17.249208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d2910-07cc-4c1d-b2af-777c0669fd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:17.249875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d2910-07cc-4c1d-b2af-777c0669fd5f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:17.250158 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d2910-07cc-4c1d-b2af-777c0669fd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:17.250423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d2910-07cc-4c1d-b2af-777c0669fd5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:17.251227 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d2910-07cc-4c1d-b2af-777c0669fd5f None admin] RBAC: Adding query filter params (role.id=485bbc11b43a4e36b7ccb740582ec8d2, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=035ae78ef3cb43e985e06cc3fa971c8b) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:17.251567 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d2910-07cc-4c1d-b2af-777c0669fd5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:17.253629 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d2910-07cc-4c1d-b2af-777c0669fd5f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:17.258508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b726596-a467-4ffe-8372-03dc470588ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:17.269705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b726596-a467-4ffe-8372-03dc470588ef None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:17.269924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 462/907] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:31:17 2022] GET /identity/v3/role_assignments?user.id=035ae78ef3cb43e985e06cc3fa971c8b&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=485bbc11b43a4e36b7ccb740582ec8d2 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:17.270010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b726596-a467-4ffe-8372-03dc470588ef None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:17.270010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b726596-a467-4ffe-8372-03dc470588ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:17.270137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7e122-6235-4017-b29e-771905999eb6 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:17.270206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b726596-a467-4ffe-8372-03dc470588ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:17.270285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7e122-6235-4017-b29e-771905999eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:17.270285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7e122-6235-4017-b29e-771905999eb6 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:17.270410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b726596-a467-4ffe-8372-03dc470588ef None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:17.270477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 463/908] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:17.270477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624a74c-0e7b-4fd9-9bd7-375c330e34c8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:17.270477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624a74c-0e7b-4fd9-9bd7-375c330e34c8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:17.270477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624a74c-0e7b-4fd9-9bd7-375c330e34c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:17.290283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a624a74c-0e7b-4fd9-9bd7-375c330e34c8 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:17.291641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a624a74c-0e7b-4fd9-9bd7-375c330e34c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:17.292184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 446/909] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:31:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:31:17.350636 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:17.396157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 464/910] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:17.415504 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b726596-a467-4ffe-8372-03dc470588ef None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:17.531375 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b726596-a467-4ffe-8372-03dc470588ef None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:17.570351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ace37f-3b15-4e99-ad1a-45adf56c306a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:17.571574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ace37f-3b15-4e99-ad1a-45adf56c306a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:17.572120 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ace37f-3b15-4e99-ad1a-45adf56c306a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:17.572394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ace37f-3b15-4e99-ad1a-45adf56c306a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:17.573038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ace37f-3b15-4e99-ad1a-45adf56c306a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:17.574441 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ace37f-3b15-4e99-ad1a-45adf56c306a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-02ace37f-3b15-4e99-ad1a-45adf56c306a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:17.588245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:17.589261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:17.590528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:17.590528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:17.590528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:17.590528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:17.590528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:17.590528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:17.590528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:17.590528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:17.590528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:17.590528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:17.590528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:17.590528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:17.590528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:17.590528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 10:31:17.590528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:17.593551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 447/911] 10.222.0.47 () {56 vars in 1132 bytes} [Wed Aug 3 10:31:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:17.596905 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624a74c-0e7b-4fd9-9bd7-375c330e34c8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:17.701662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624a74c-0e7b-4fd9-9bd7-375c330e34c8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:17.738046 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8001e-1572-4096-be9b-0ffdcc547a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:17.738676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8001e-1572-4096-be9b-0ffdcc547a79 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:17.738952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8001e-1572-4096-be9b-0ffdcc547a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:17.739224 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8001e-1572-4096-be9b-0ffdcc547a79 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:17.739923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8001e-1572-4096-be9b-0ffdcc547a79 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:17.740223 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8001e-1572-4096-be9b-0ffdcc547a79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:17.741604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8001e-1572-4096-be9b-0ffdcc547a79 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:17.751324 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 465/912] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 3 10:31:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:17.754559 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ace37f-3b15-4e99-ad1a-45adf56c306a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:17.778235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:17.864947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ace37f-3b15-4e99-ad1a-45adf56c306a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:17.899578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190ca5d-ef41-4fea-a47b-2d444d7deaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:17.900203 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190ca5d-ef41-4fea-a47b-2d444d7deaf5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:17.900479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190ca5d-ef41-4fea-a47b-2d444d7deaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:17.900749 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190ca5d-ef41-4fea-a47b-2d444d7deaf5 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:17.901426 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190ca5d-ef41-4fea-a47b-2d444d7deaf5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:17.901819 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190ca5d-ef41-4fea-a47b-2d444d7deaf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:17.903179 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190ca5d-ef41-4fea-a47b-2d444d7deaf5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:17.914870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 448/913] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 3 10:31:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:17.916673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8001e-1572-4096-be9b-0ffdcc547a79 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:17.953813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:17.971272 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861110ab-c02e-49b1-8ddb-90f8263c7b7b None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:17.971584 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861110ab-c02e-49b1-8ddb-90f8263c7b7b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:17.971824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861110ab-c02e-49b1-8ddb-90f8263c7b7b None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:17.973332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 449/914] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:31:17.978612 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19917151-f395-4b46-ab05-fcc7832f0b8b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:17.978890 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19917151-f395-4b46-ab05-fcc7832f0b8b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:17.979162 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19917151-f395-4b46-ab05-fcc7832f0b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:18.002482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-19917151-f395-4b46-ab05-fcc7832f0b8b None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:18.003685 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-19917151-f395-4b46-ab05-fcc7832f0b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:18.029873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8001e-1572-4096-be9b-0ffdcc547a79 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:18.068091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c18be-39ef-42ad-abc0-c887e8f51b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:18.069113 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c18be-39ef-42ad-abc0-c887e8f51b2c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:18.069480 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c18be-39ef-42ad-abc0-c887e8f51b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:18.069853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c18be-39ef-42ad-abc0-c887e8f51b2c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:18.070736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c18be-39ef-42ad-abc0-c887e8f51b2c None admin] RBAC: Adding query filter params (interface=public, service_id=5f5492bc7de449ceaff9f4ce8e2e832e, region_id=RegionOne) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:18.071117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c18be-39ef-42ad-abc0-c887e8f51b2c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:18.072587 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c18be-39ef-42ad-abc0-c887e8f51b2c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:18.081378 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 466/915] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 3 10:31:17 2022] GET /identity/v3/endpoints?service_id=5f5492bc7de449ceaff9f4ce8e2e832e&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:18.084780 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 450/916] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:18.086715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c18be-39ef-42ad-abc0-c887e8f51b2c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:18.111345 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19917151-f395-4b46-ab05-fcc7832f0b8b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:18.131195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:18.206440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:18.209893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c18be-39ef-42ad-abc0-c887e8f51b2c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:18.251064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19917151-f395-4b46-ab05-fcc7832f0b8b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:18.254898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9191ec-0a0c-41f4-840f-97d3371e1e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:18.255700 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9191ec-0a0c-41f4-840f-97d3371e1e1b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:18.256128 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9191ec-0a0c-41f4-840f-97d3371e1e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:18.256566 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9191ec-0a0c-41f4-840f-97d3371e1e1b None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:18.257390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9191ec-0a0c-41f4-840f-97d3371e1e1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:18.258972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9191ec-0a0c-41f4-840f-97d3371e1e1b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:18.266286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 467/917] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 3 10:31:18 2022] GET /identity/v3/services => generated 1268 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:31:18.289854 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea41be-7239-45cf-aa06-634b94897581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:18.290678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea41be-7239-45cf-aa06-634b94897581 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:18.291063 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea41be-7239-45cf-aa06-634b94897581 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:18.291436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea41be-7239-45cf-aa06-634b94897581 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:18.292271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea41be-7239-45cf-aa06-634b94897581 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:18.293913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea41be-7239-45cf-aa06-634b94897581 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:18.313232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 451/918] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 3 10:31:18 2022] POST /identity/v3/services => generated 241 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:31:19.445546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f103d-16c7-4b9b-9283-f88d70d48e6a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:19.446547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f103d-16c7-4b9b-9283-f88d70d48e6a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:19.446829 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f103d-16c7-4b9b-9283-f88d70d48e6a None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:19.448383 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 468/919] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:19.455427 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623acff2-b52a-4d69-8c30-139c04c8a155 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:19.456542 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623acff2-b52a-4d69-8c30-139c04c8a155 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:19.457034 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623acff2-b52a-4d69-8c30-139c04c8a155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:19.477779 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-623acff2-b52a-4d69-8c30-139c04c8a155 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:19.478980 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-623acff2-b52a-4d69-8c30-139c04c8a155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:19.537110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4423fb1-c086-42d5-8621-dd2c9afdf8ce None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:19.537781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4423fb1-c086-42d5-8621-dd2c9afdf8ce None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:19.538265 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4423fb1-c086-42d5-8621-dd2c9afdf8ce None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:19.540064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 469/920] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:19.545634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b22e26-ee49-49a0-b5cc-ba2bee209adf None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:19.546220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b22e26-ee49-49a0-b5cc-ba2bee209adf None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:19.546784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b22e26-ee49-49a0-b5cc-ba2bee209adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:19.554965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:19.561523 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 452/921] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:19.571994 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b4b22e26-ee49-49a0-b5cc-ba2bee209adf None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:19.573433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b4b22e26-ee49-49a0-b5cc-ba2bee209adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:19.583527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623acff2-b52a-4d69-8c30-139c04c8a155 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:19.665663 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 470/922] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:19.682009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b22e26-ee49-49a0-b5cc-ba2bee209adf None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:19.713621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623acff2-b52a-4d69-8c30-139c04c8a155 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:19.757941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e15f17-1b85-4196-9aae-b515b0847c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:19.759060 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e15f17-1b85-4196-9aae-b515b0847c89 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:19.759590 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e15f17-1b85-4196-9aae-b515b0847c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:19.760061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e15f17-1b85-4196-9aae-b515b0847c89 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:19.775261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e15f17-1b85-4196-9aae-b515b0847c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-c4e15f17-1b85-4196-9aae-b515b0847c89 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:31:19.782323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:19.783897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:19.784801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:19.784801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:19.784801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:19.784801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:19.784801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:31:19.784801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:31:19.784801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:31:19.784801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:31:19.784801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:31:19.784801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:31:19.784801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 10:31:19.784801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:19.786946 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 453/923] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Aug 3 10:31:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:19.793772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e15f17-1b85-4196-9aae-b515b0847c89 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:19.812336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:19.822265 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b22e26-ee49-49a0-b5cc-ba2bee209adf None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:19.856309 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:19.865874 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fcd07-d34d-426d-978d-040aef09dade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:19.866942 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fcd07-d34d-426d-978d-040aef09dade None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:19.867552 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fcd07-d34d-426d-978d-040aef09dade None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:19.868047 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fcd07-d34d-426d-978d-040aef09dade None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:19.869023 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fcd07-d34d-426d-978d-040aef09dade None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:19.870706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fcd07-d34d-426d-978d-040aef09dade None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:19.873721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: WARNING keystone.server.flask.common [None req-348fcd07-d34d-426d-978d-040aef09dade None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 03 10:31:19.894396 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.sql.core [None req-348fcd07-d34d-426d-978d-040aef09dade None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 03 10:31:19.894396 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 03 10:31:19.894396 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [parameters: {'id': 'f2f39acf12c5478a94006be083e7efe2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 03 10:31:19.894396 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80318) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 03 10:31:19.899565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e15f17-1b85-4196-9aae-b515b0847c89 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:19.903627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-348fcd07-d34d-426d-978d-040aef09dade None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f2f39acf12c5478a94006be083e7efe2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f2f39acf12c5478a94006be083e7efe2. Aug 03 10:31:19.903627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:19.903627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 10:31:19.903627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 10:31:19.903627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 10:31:19.903627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 10:31:19.903627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 10:31:19.903627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 10:31:19.903627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 10:31:19.903627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application conn.query(q) Aug 03 10:31:19.903627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 10:31:19.903627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 10:31:19.903627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result.read() Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 03 10:31:19.904444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application next(self.gen) Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application yield resource Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application next(self.gen) Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.session.rollback() Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.force_reraise() Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise self.value Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application session.commit() Aug 03 10:31:19.905481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._prepare_impl() Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.session.flush() Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._flush(objects) Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application compat.raise_( Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application flush_context.execute() Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 03 10:31:19.906398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rec.execute(self) Aug 03 10:31:19.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 03 10:31:19.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 03 10:31:19.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 03 10:31:19.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 03 10:31:19.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 03 10:31:19.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 03 10:31:19.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 03 10:31:19.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 03 10:31:19.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 03 10:31:19.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 03 10:31:19.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 03 10:31:19.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 03 10:31:19.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 03 10:31:19.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 03 10:31:19.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application conn.query(q) Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result.read() Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 10:31:19.908147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 10:31:19.910183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 10:31:19.910183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 10:31:19.910183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 10:31:19.910183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 10:31:19.910183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 10:31:19.910183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 10:31:19.910183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 03 10:31:19.910183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 03 10:31:19.910183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [parameters: {'id': 'f2f39acf12c5478a94006be083e7efe2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 03 10:31:19.910183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 03 10:31:19.910183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:19.910183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:19.910183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:19.910183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:19.910183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:19.912111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:19.914314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:19.914314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 03 10:31:19.914314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 03 10:31:19.914314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:19.914314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:19.914314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 03 10:31:19.914314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 03 10:31:19.914314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f2f39acf12c5478a94006be083e7efe2. Aug 03 10:31:19.914314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:19.920046 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 471/924] 10.222.0.47 () {60 vars in 1140 bytes} [Wed Aug 3 10:31:19 2022] POST /identity/v3/projects => generated 262 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:19.921692 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fcd07-d34d-426d-978d-040aef09dade None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:19.948890 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fc4a0-3352-4faa-a488-600f5bab85fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:19.950222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fc4a0-3352-4faa-a488-600f5bab85fa None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:19.950508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fc4a0-3352-4faa-a488-600f5bab85fa None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:19.950786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fc4a0-3352-4faa-a488-600f5bab85fa None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:19.951517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fc4a0-3352-4faa-a488-600f5bab85fa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:19.951823 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fc4a0-3352-4faa-a488-600f5bab85fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:19.953206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fc4a0-3352-4faa-a488-600f5bab85fa None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:19.967889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 454/925] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Aug 3 10:31:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:19.968768 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fc4a0-3352-4faa-a488-600f5bab85fa None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:20.045310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fcd07-d34d-426d-978d-040aef09dade None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:20.073879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fc4a0-3352-4faa-a488-600f5bab85fa None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:20.085368 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c7e34-afc6-4793-b252-496f3200b71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:20.086419 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c7e34-afc6-4793-b252-496f3200b71b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:20.086918 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c7e34-afc6-4793-b252-496f3200b71b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:20.087534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c7e34-afc6-4793-b252-496f3200b71b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:20.096265 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c7e34-afc6-4793-b252-496f3200b71b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:20.098353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:20.098909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c7e34-afc6-4793-b252-496f3200b71b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:20.110660 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b9cb8-4fc3-4fe8-a472-a7971968f612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:20.111346 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b9cb8-4fc3-4fe8-a472-a7971968f612 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:20.111627 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b9cb8-4fc3-4fe8-a472-a7971968f612 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:20.111946 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b9cb8-4fc3-4fe8-a472-a7971968f612 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:20.115459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-834c7e34-afc6-4793-b252-496f3200b71b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:20.115459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:20.115459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:20.115459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:20.115459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:20.115459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:20.115459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:20.115459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:20.115459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:20.115459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:20.115459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:20.115459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:20.115459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:31:20.115459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:31:20.115459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:20.116403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:20.117335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:31:20.117335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:31:20.117335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:31:20.117335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:31:20.117335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:31:20.117335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:31:20.117335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:20.117335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:20.119952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 472/926] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:20.125273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c7e34-afc6-4793-b252-496f3200b71b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:20.130272 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b9cb8-4fc3-4fe8-a472-a7971968f612 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:20.132953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b9cb8-4fc3-4fe8-a472-a7971968f612 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:20.144847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:20.154693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-1d9b9cb8-4fc3-4fe8-a472-a7971968f612 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:20.154693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:20.154693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:20.154693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:20.154693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:20.154693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:20.154693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:20.154693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:20.154693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:20.154693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:20.154693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:20.154693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:20.154693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:20.154693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:20.154693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:20.155666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:20.156613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:20.156613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:20.156613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:20.156613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:20.161217 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 455/927] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:20.162150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b9cb8-4fc3-4fe8-a472-a7971968f612 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:20.274005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c7e34-afc6-4793-b252-496f3200b71b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:20.297683 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b9cb8-4fc3-4fe8-a472-a7971968f612 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:20.315519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4b48b-9a36-470a-a90b-7f9485d81b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:20.316152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4b48b-9a36-470a-a90b-7f9485d81b68 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:20.316436 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4b48b-9a36-470a-a90b-7f9485d81b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:20.316714 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4b48b-9a36-470a-a90b-7f9485d81b68 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:20.317730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4b48b-9a36-470a-a90b-7f9485d81b68 None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:20.318045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4b48b-9a36-470a-a90b-7f9485d81b68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:20.319878 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4b48b-9a36-470a-a90b-7f9485d81b68 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:20.337053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4537f6-2521-4ca5-9e5b-5bcad2923eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:20.338051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4537f6-2521-4ca5-9e5b-5bcad2923eaa None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:20.338553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4537f6-2521-4ca5-9e5b-5bcad2923eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:20.339039 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4537f6-2521-4ca5-9e5b-5bcad2923eaa None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:20.340192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4537f6-2521-4ca5-9e5b-5bcad2923eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:20.340790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4537f6-2521-4ca5-9e5b-5bcad2923eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:20.342480 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4537f6-2521-4ca5-9e5b-5bcad2923eaa None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:20.353173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 473/928] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 3 10:31:20 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:20.358466 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 456/929] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:20.364897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4b48b-9a36-470a-a90b-7f9485d81b68 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:20.422202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbf4f1-5e90-4b1c-a96a-18a9a9317878 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:20.422787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbf4f1-5e90-4b1c-a96a-18a9a9317878 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:20.423083 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbf4f1-5e90-4b1c-a96a-18a9a9317878 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:20.425568 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 457/930] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:20.431395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f844d13-c505-483d-88c5-7c1f296b41b7 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:20.431694 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f844d13-c505-483d-88c5-7c1f296b41b7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:20.431940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f844d13-c505-483d-88c5-7c1f296b41b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:20.451356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7f844d13-c505-483d-88c5-7c1f296b41b7 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:20.452587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7f844d13-c505-483d-88c5-7c1f296b41b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:20.454872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:20.483547 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:20.492028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4b48b-9a36-470a-a90b-7f9485d81b68 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:20.532296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 458/931] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:20.532542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3b78b-bdf5-427a-934c-6b2878e0aea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:20.533820 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8a39e-cdb5-4dd1-9c3e-6142770a9650 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:20.533820 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8a39e-cdb5-4dd1-9c3e-6142770a9650 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:20.534115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8a39e-cdb5-4dd1-9c3e-6142770a9650 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:20.536560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 459/932] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:20.540950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c9afa-942d-4aac-8cda-dd4367a261ef None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:20.541238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c9afa-942d-4aac-8cda-dd4367a261ef None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:20.541591 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c9afa-942d-4aac-8cda-dd4367a261ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:20.542873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3b78b-bdf5-427a-934c-6b2878e0aea3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:20.543364 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3b78b-bdf5-427a-934c-6b2878e0aea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:20.543645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3b78b-bdf5-427a-934c-6b2878e0aea3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:20.552731 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3b78b-bdf5-427a-934c-6b2878e0aea3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:20.554212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3b78b-bdf5-427a-934c-6b2878e0aea3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:20.560461 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e10c9afa-942d-4aac-8cda-dd4367a261ef None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:20.561691 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e10c9afa-942d-4aac-8cda-dd4367a261ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-e8e3b78b-bdf5-427a-934c-6b2878e0aea3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:31:20.566763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:20.567701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:20.569039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:20.569039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:20.569039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:31:20.569039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:31:20.569039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:31:20.569039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:31:20.569039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:31:20.569039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:31:20.569039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 10:31:20.569039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:20.569039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 474/933] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 3 10:31:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:20.569039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3b78b-bdf5-427a-934c-6b2878e0aea3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:20.639242 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 460/934] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:20.640096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c9afa-942d-4aac-8cda-dd4367a261ef None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:20.680096 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3b78b-bdf5-427a-934c-6b2878e0aea3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:20.708926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:20.716295 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47d7fa-3a1f-4a47-ae25-7e6a19fc13e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:20.716991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47d7fa-3a1f-4a47-ae25-7e6a19fc13e5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:20.717271 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47d7fa-3a1f-4a47-ae25-7e6a19fc13e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:20.717575 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47d7fa-3a1f-4a47-ae25-7e6a19fc13e5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:20.718213 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47d7fa-3a1f-4a47-ae25-7e6a19fc13e5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:20.719574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47d7fa-3a1f-4a47-ae25-7e6a19fc13e5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-2d47d7fa-3a1f-4a47-ae25-7e6a19fc13e5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:20.729182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:20.730451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:20.731641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:20.731641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:20.731641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:20.731641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:20.731641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:20.731641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:20.731641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:20.731641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:20.731641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:20.731641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:20.731641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:20.731641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:20.731641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 10:31:20.731641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:20.731641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 475/935] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 3 10:31:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:20.732614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47d7fa-3a1f-4a47-ae25-7e6a19fc13e5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:20.760764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:20.768499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c9afa-942d-4aac-8cda-dd4367a261ef None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:20.809668 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2988dcb-1f25-41e1-a83d-c6ff0f676dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:20.810226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2988dcb-1f25-41e1-a83d-c6ff0f676dec None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:20.810510 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2988dcb-1f25-41e1-a83d-c6ff0f676dec None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:20.810818 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2988dcb-1f25-41e1-a83d-c6ff0f676dec None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:20.812209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2988dcb-1f25-41e1-a83d-c6ff0f676dec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:20.812209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2988dcb-1f25-41e1-a83d-c6ff0f676dec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:20.813315 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2988dcb-1f25-41e1-a83d-c6ff0f676dec None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:20.831958 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 461/936] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Aug 3 10:31:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:20.832773 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2988dcb-1f25-41e1-a83d-c6ff0f676dec None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:20.847064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47d7fa-3a1f-4a47-ae25-7e6a19fc13e5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:20.882963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df759fc0-d2b2-44ed-8d1e-4791ddd8ff2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:20.883624 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df759fc0-d2b2-44ed-8d1e-4791ddd8ff2c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:20.883909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df759fc0-d2b2-44ed-8d1e-4791ddd8ff2c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:20.884443 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df759fc0-d2b2-44ed-8d1e-4791ddd8ff2c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:20.884852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df759fc0-d2b2-44ed-8d1e-4791ddd8ff2c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:20.886257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df759fc0-d2b2-44ed-8d1e-4791ddd8ff2c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-df759fc0-d2b2-44ed-8d1e-4791ddd8ff2c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:20.895712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:20.896696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:20.897564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 476/937] 10.222.0.47 () {56 vars in 1132 bytes} [Wed Aug 3 10:31:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:20.899121 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df759fc0-d2b2-44ed-8d1e-4791ddd8ff2c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:20.949343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2988dcb-1f25-41e1-a83d-c6ff0f676dec None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:20.966659 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:20.987651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92e54f-0357-4123-9a17-506d5a4c1156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:20.988284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92e54f-0357-4123-9a17-506d5a4c1156 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:20.989030 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92e54f-0357-4123-9a17-506d5a4c1156 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:20.989310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92e54f-0357-4123-9a17-506d5a4c1156 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:20.990090 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92e54f-0357-4123-9a17-506d5a4c1156 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:20.990705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92e54f-0357-4123-9a17-506d5a4c1156 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:20.992142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92e54f-0357-4123-9a17-506d5a4c1156 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:21.001749 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 462/938] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 3 10:31:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:21.002647 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92e54f-0357-4123-9a17-506d5a4c1156 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:21.012331 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df759fc0-d2b2-44ed-8d1e-4791ddd8ff2c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:21.021803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:21.051653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40590516-a240-4df6-969c-3d330ea0fa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:21.052626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40590516-a240-4df6-969c-3d330ea0fa64 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:21.052916 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40590516-a240-4df6-969c-3d330ea0fa64 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:21.053197 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40590516-a240-4df6-969c-3d330ea0fa64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:21.065068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40590516-a240-4df6-969c-3d330ea0fa64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:21.066474 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40590516-a240-4df6-969c-3d330ea0fa64 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:21.081266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-40590516-a240-4df6-969c-3d330ea0fa64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:21.081266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:21.081266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:21.081266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:21.081266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:21.081266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:21.081266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:21.081266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:21.081266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:21.081266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:21.081266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:21.081266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:21.081266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:21.081266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:21.081266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:21.082345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:21.083276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:21.083276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:21.083276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:21.083276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:21.083276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 477/939] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:21.083276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40590516-a240-4df6-969c-3d330ea0fa64 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:21.118966 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92e54f-0357-4123-9a17-506d5a4c1156 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:21.155970 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31036bc-890d-48bb-9ea6-71079c86e96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:21.156614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31036bc-890d-48bb-9ea6-71079c86e96e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:21.156900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31036bc-890d-48bb-9ea6-71079c86e96e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:21.157180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31036bc-890d-48bb-9ea6-71079c86e96e None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:21.158487 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31036bc-890d-48bb-9ea6-71079c86e96e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:21.160031 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31036bc-890d-48bb-9ea6-71079c86e96e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:21.160273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31036bc-890d-48bb-9ea6-71079c86e96e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:21.167574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 463/940] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 3 10:31:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:21.168425 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31036bc-890d-48bb-9ea6-71079c86e96e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:21.207394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40590516-a240-4df6-969c-3d330ea0fa64 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:21.218353 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:21.244187 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e7fdb-add5-4c9c-b4d7-fe2f4f47b1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:21.244819 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e7fdb-add5-4c9c-b4d7-fe2f4f47b1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:21.245094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e7fdb-add5-4c9c-b4d7-fe2f4f47b1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:21.245370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e7fdb-add5-4c9c-b4d7-fe2f4f47b1f8 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:21.246099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e7fdb-add5-4c9c-b4d7-fe2f4f47b1f8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:21.246403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e7fdb-add5-4c9c-b4d7-fe2f4f47b1f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:21.247755 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e7fdb-add5-4c9c-b4d7-fe2f4f47b1f8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:21.253069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 478/941] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 3 10:31:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:21.255109 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e7fdb-add5-4c9c-b4d7-fe2f4f47b1f8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:21.257775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:21.279312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31036bc-890d-48bb-9ea6-71079c86e96e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:21.317015 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27e55e-0b9f-495a-b0fa-27d3eadb6d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:21.317603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27e55e-0b9f-495a-b0fa-27d3eadb6d3d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:21.317969 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27e55e-0b9f-495a-b0fa-27d3eadb6d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:21.318150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27e55e-0b9f-495a-b0fa-27d3eadb6d3d None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:21.318843 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27e55e-0b9f-495a-b0fa-27d3eadb6d3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:21.319148 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27e55e-0b9f-495a-b0fa-27d3eadb6d3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:21.320517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27e55e-0b9f-495a-b0fa-27d3eadb6d3d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:21.338091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 464/942] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:21.338091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27e55e-0b9f-495a-b0fa-27d3eadb6d3d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:21.365349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e7fdb-add5-4c9c-b4d7-fe2f4f47b1f8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:21.402605 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b727c0-dfaa-4784-93de-059f442e3d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:21.403244 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b727c0-dfaa-4784-93de-059f442e3d4b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:21.403527 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b727c0-dfaa-4784-93de-059f442e3d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:21.403800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b727c0-dfaa-4784-93de-059f442e3d4b None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:21.404516 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b727c0-dfaa-4784-93de-059f442e3d4b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:21.404798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b727c0-dfaa-4784-93de-059f442e3d4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:21.406192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b727c0-dfaa-4784-93de-059f442e3d4b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:21.411460 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 479/943] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 3 10:31:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:21.412310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b727c0-dfaa-4784-93de-059f442e3d4b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:21.452711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27e55e-0b9f-495a-b0fa-27d3eadb6d3d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:21.453826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:21.488632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404be52f-e293-4c4d-90ed-19601bdc9d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:21.489907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404be52f-e293-4c4d-90ed-19601bdc9d9c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:21.490186 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404be52f-e293-4c4d-90ed-19601bdc9d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:21.490643 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404be52f-e293-4c4d-90ed-19601bdc9d9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:21.491406 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404be52f-e293-4c4d-90ed-19601bdc9d9c None admin] RBAC: Adding query filter params (interface=public, service_id=e9f64f52b5c14b94928b300afd795411, region_id=RegionOne) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:21.491718 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404be52f-e293-4c4d-90ed-19601bdc9d9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:21.493083 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404be52f-e293-4c4d-90ed-19601bdc9d9c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:21.498496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 465/944] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 3 10:31:21 2022] GET /identity/v3/endpoints?service_id=e9f64f52b5c14b94928b300afd795411&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:21.504211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404be52f-e293-4c4d-90ed-19601bdc9d9c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:21.509228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:21.522596 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b727c0-dfaa-4784-93de-059f442e3d4b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:21.560077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489135a-a8ce-4dcd-92a6-a141ba5f1e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:21.561130 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489135a-a8ce-4dcd-92a6-a141ba5f1e09 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:21.561667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489135a-a8ce-4dcd-92a6-a141ba5f1e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:21.562044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489135a-a8ce-4dcd-92a6-a141ba5f1e09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:21.570113 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489135a-a8ce-4dcd-92a6-a141ba5f1e09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:21.571625 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489135a-a8ce-4dcd-92a6-a141ba5f1e09 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:21.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-2489135a-a8ce-4dcd-92a6-a141ba5f1e09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:21.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:21.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:21.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:21.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:21.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:21.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:21.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:21.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:21.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:21.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:21.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:21.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:31:21.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:31:21.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:21.585964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:21.592278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:31:21.592278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:31:21.592278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:31:21.592278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:31:21.592278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:31:21.592278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:31:21.592278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:21.592278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:21.592278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 480/945] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:21.592278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489135a-a8ce-4dcd-92a6-a141ba5f1e09 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:21.630046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404be52f-e293-4c4d-90ed-19601bdc9d9c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:21.667263 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e198985-6d6b-47d7-880b-e1532a9da1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:21.667891 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e198985-6d6b-47d7-880b-e1532a9da1f9 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:21.668179 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e198985-6d6b-47d7-880b-e1532a9da1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:21.668450 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e198985-6d6b-47d7-880b-e1532a9da1f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:21.669172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e198985-6d6b-47d7-880b-e1532a9da1f9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:21.670588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e198985-6d6b-47d7-880b-e1532a9da1f9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:21.705493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 466/946] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:31:21 2022] POST /identity/v3/endpoints => generated 347 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:31:21.705493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e198985-6d6b-47d7-880b-e1532a9da1f9 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:21.708765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:21.712877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489135a-a8ce-4dcd-92a6-a141ba5f1e09 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:21.767243 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01ef65-566c-4ec8-9acd-057d7b9c88d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:21.767243 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01ef65-566c-4ec8-9acd-057d7b9c88d0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:21.767569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01ef65-566c-4ec8-9acd-057d7b9c88d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:21.767828 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01ef65-566c-4ec8-9acd-057d7b9c88d0 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:21.768426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01ef65-566c-4ec8-9acd-057d7b9c88d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:21.769860 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01ef65-566c-4ec8-9acd-057d7b9c88d0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:21.785970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 481/947] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 3 10:31:21 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:31:21.803422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:21.859788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e198985-6d6b-47d7-880b-e1532a9da1f9 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:21.904913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f56d7-e7e1-4039-b157-29149ce91432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:21.905606 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f56d7-e7e1-4039-b157-29149ce91432 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:21.905890 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f56d7-e7e1-4039-b157-29149ce91432 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:21.906165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f56d7-e7e1-4039-b157-29149ce91432 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:21.907001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f56d7-e7e1-4039-b157-29149ce91432 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:21.907268 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f56d7-e7e1-4039-b157-29149ce91432 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:21.909238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f56d7-e7e1-4039-b157-29149ce91432 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:21.932410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 467/948] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:31:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:21.938421 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01ef65-566c-4ec8-9acd-057d7b9c88d0 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:22.060622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01ef65-566c-4ec8-9acd-057d7b9c88d0 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:22.100381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27358ea-8543-4484-b157-f949ef9692ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:22.101125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27358ea-8543-4484-b157-f949ef9692ce None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:22.101508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27358ea-8543-4484-b157-f949ef9692ce None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:22.101775 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27358ea-8543-4484-b157-f949ef9692ce None admin] PATH_INFO: `/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/035ae78ef3cb43e985e06cc3fa971c8b/roles/485bbc11b43a4e36b7ccb740582ec8d2` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:22.104082 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27358ea-8543-4484-b157-f949ef9692ce None admin] RBAC: Authorizing `identity:create_grant(project_id=4272b859979d49c8b8f0c17ec9eb81bc, user_id=035ae78ef3cb43e985e06cc3fa971c8b, role_id=485bbc11b43a4e36b7ccb740582ec8d2)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:22.105536 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27358ea-8543-4484-b157-f949ef9692ce None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:22.123911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 482/949] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:31:21 2022] PUT /identity/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/035ae78ef3cb43e985e06cc3fa971c8b/roles/485bbc11b43a4e36b7ccb740582ec8d2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:31:22.496972 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680daeff-1ed9-4d5e-87e6-8b0acb39706d None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:22.497554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680daeff-1ed9-4d5e-87e6-8b0acb39706d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:22.497554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680daeff-1ed9-4d5e-87e6-8b0acb39706d None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:22.499207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 468/950] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:22.505190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b73e9-7351-4979-b391-0651b97a101a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:22.505486 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b73e9-7351-4979-b391-0651b97a101a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:22.505775 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b73e9-7351-4979-b391-0651b97a101a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:22.525257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-262b73e9-7351-4979-b391-0651b97a101a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:22.526990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-262b73e9-7351-4979-b391-0651b97a101a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:22.541673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:22.600202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 483/951] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:22.624198 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680daeff-1ed9-4d5e-87e6-8b0acb39706d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:22.744081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680daeff-1ed9-4d5e-87e6-8b0acb39706d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:22.765357 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:22.780783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54db8e-9fae-4195-bbe6-25323ead4fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:22.781412 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54db8e-9fae-4195-bbe6-25323ead4fa0 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:22.781799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54db8e-9fae-4195-bbe6-25323ead4fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:22.782078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54db8e-9fae-4195-bbe6-25323ead4fa0 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:22.782795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54db8e-9fae-4195-bbe6-25323ead4fa0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:22.784775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54db8e-9fae-4195-bbe6-25323ead4fa0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:22.787412 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: WARNING keystone.server.flask.common [None req-4c54db8e-9fae-4195-bbe6-25323ead4fa0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 03 10:31:22.800533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.sql.core [None req-4c54db8e-9fae-4195-bbe6-25323ead4fa0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 03 10:31:22.800533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 03 10:31:22.800533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [parameters: {'user_id': '680713e70abe40f8b22f05a9bf74459b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 03 10:31:22.800533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80319) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 03 10:31:22.818033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-4c54db8e-9fae-4195-bbe6-25323ead4fa0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 03 10:31:22.818033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:22.818033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 10:31:22.818033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 10:31:22.818033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 10:31:22.818033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 10:31:22.818033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 10:31:22.818033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 10:31:22.818033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 10:31:22.818033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application conn.query(q) Aug 03 10:31:22.818033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 10:31:22.818033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 10:31:22.818033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 10:31:22.818033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application result.read() Aug 03 10:31:22.818033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 10:31:22.819199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application next(self.gen) Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application yield resource Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application next(self.gen) Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self.session.rollback() Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self.force_reraise() Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise self.value Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application session.commit() Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 03 10:31:22.821172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._prepare_impl() Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self.session.flush() Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._flush(objects) Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application compat.raise_( Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application flush_context.execute() Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rec.execute(self) Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 03 10:31:22.822394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 10:31:22.823549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application conn.query(q) Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application result.read() Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 10:31:22.828754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 10:31:22.831013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 10:31:22.831013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 10:31:22.831013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 10:31:22.831013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 10:31:22.831013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 03 10:31:22.831013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 03 10:31:22.831013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [parameters: {'user_id': '680713e70abe40f8b22f05a9bf74459b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 03 10:31:22.831013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 03 10:31:22.831013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:22.831013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:22.831013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:22.831013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:22.831013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:22.831013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:22.831013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 03 10:31:22.832973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 03 10:31:22.835174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 03 10:31:22.835174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 03 10:31:22.835174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 03 10:31:22.835174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 03 10:31:22.835174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 03 10:31:22.835174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:22.835174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 469/952] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Aug 3 10:31:22 2022] POST /identity/v3/users => generated 160 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:22.836088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b73e9-7351-4979-b391-0651b97a101a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:22.930738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b73e9-7351-4979-b391-0651b97a101a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:22.974464 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cb254-34c1-4dbe-abf7-b4c7543331a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:22.975134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cb254-34c1-4dbe-abf7-b4c7543331a8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:22.975412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cb254-34c1-4dbe-abf7-b4c7543331a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:22.975682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cb254-34c1-4dbe-abf7-b4c7543331a8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:22.984105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cb254-34c1-4dbe-abf7-b4c7543331a8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:22.985687 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cb254-34c1-4dbe-abf7-b4c7543331a8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-fe6cb254-34c1-4dbe-abf7-b4c7543331a8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:31:22.998660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:23.000947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:23.003866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:23.003866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:23.003866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:31:23.003866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:31:23.003866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:31:23.003866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:31:23.003866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:31:23.003866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:31:23.003866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 10:31:23.003866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:23.003866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 484/953] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:31:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:23.009710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54db8e-9fae-4195-bbe6-25323ead4fa0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:23.121951 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54db8e-9fae-4195-bbe6-25323ead4fa0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:23.159309 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9b48a-ebf6-4f0e-91d5-e71e015c5cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:23.159938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9b48a-ebf6-4f0e-91d5-e71e015c5cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:23.160223 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9b48a-ebf6-4f0e-91d5-e71e015c5cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:23.160498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9b48a-ebf6-4f0e-91d5-e71e015c5cb7 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:23.161258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9b48a-ebf6-4f0e-91d5-e71e015c5cb7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:23.161600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9b48a-ebf6-4f0e-91d5-e71e015c5cb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:23.163232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9b48a-ebf6-4f0e-91d5-e71e015c5cb7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:23.206175 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 470/954] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 3 10:31:23 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:23.910378 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e77bf-3fc1-4756-b5e1-b7d3092a48f0 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:23.910964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e77bf-3fc1-4756-b5e1-b7d3092a48f0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:23.910964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e77bf-3fc1-4756-b5e1-b7d3092a48f0 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:23.912399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 485/955] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:23.917644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b71e1-ba7c-4151-9c40-01b0a9a7c22e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:23.918096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b71e1-ba7c-4151-9c40-01b0a9a7c22e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:23.918179 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b71e1-ba7c-4151-9c40-01b0a9a7c22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:23.939557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f12b71e1-ba7c-4151-9c40-01b0a9a7c22e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:23.940993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f12b71e1-ba7c-4151-9c40-01b0a9a7c22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:23.952738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611f302-bc4a-4fc5-9d6f-0af63737080f None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:23.953027 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611f302-bc4a-4fc5-9d6f-0af63737080f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:23.953448 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611f302-bc4a-4fc5-9d6f-0af63737080f None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:23.958909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 486/956] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:31:23.960210 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911084fc-2b26-4f0a-9ade-a89940166b91 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:23.960496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911084fc-2b26-4f0a-9ade-a89940166b91 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:23.960773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911084fc-2b26-4f0a-9ade-a89940166b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:23.979958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-911084fc-2b26-4f0a-9ade-a89940166b91 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:23.981241 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-911084fc-2b26-4f0a-9ade-a89940166b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:23.982787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:23.983412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:24.020122 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 471/957] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:24.042485 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b71e1-ba7c-4151-9c40-01b0a9a7c22e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:24.061225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 487/958] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:24.084222 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911084fc-2b26-4f0a-9ade-a89940166b91 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:24.145275 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b71e1-ba7c-4151-9c40-01b0a9a7c22e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:24.179308 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc52a6c-92c4-40ec-b1ed-8c3f1047ff3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:24.179974 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc52a6c-92c4-40ec-b1ed-8c3f1047ff3b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:24.180259 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc52a6c-92c4-40ec-b1ed-8c3f1047ff3b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:24.180538 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc52a6c-92c4-40ec-b1ed-8c3f1047ff3b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:24.181276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc52a6c-92c4-40ec-b1ed-8c3f1047ff3b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:24.182688 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc52a6c-92c4-40ec-b1ed-8c3f1047ff3b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:24.187179 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911084fc-2b26-4f0a-9ade-a89940166b91 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-0dc52a6c-92c4-40ec-b1ed-8c3f1047ff3b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:24.191938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:24.194304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:24.200222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:24.200222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:24.200222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:24.200222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:24.200222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:24.200222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:24.200222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:24.200222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:24.200222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:24.200222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:24.200222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:24.200222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:24.200222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 10:31:24.200222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:24.200222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 472/959] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:24.201828 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceba40a-8228-43c6-98de-da99dfc4c12e None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:24.201828 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceba40a-8228-43c6-98de-da99dfc4c12e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:24.201828 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceba40a-8228-43c6-98de-da99dfc4c12e None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:24.201828 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 473/960] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:24.201828 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceba40a-8228-43c6-98de-da99dfc4c12e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:24.227695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b375d-722b-4079-aa4a-07f69c3ae3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:24.228341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b375d-722b-4079-aa4a-07f69c3ae3fd None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:24.228618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b375d-722b-4079-aa4a-07f69c3ae3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:24.228896 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b375d-722b-4079-aa4a-07f69c3ae3fd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:24.229564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b375d-722b-4079-aa4a-07f69c3ae3fd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:24.230919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b375d-722b-4079-aa4a-07f69c3ae3fd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:24.236296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-413b375d-722b-4079-aa4a-07f69c3ae3fd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:24.240001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:24.241390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:24.242558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:24.242558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:24.242558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:24.242558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:24.242558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:24.242558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:24.242558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:24.242558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:24.242558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:24.242558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:24.242558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:24.242558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:24.242558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 10:31:24.242558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:24.242558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 488/961] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 3 10:31:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:24.243695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340821d-7e3f-4e1a-9cfb-7ead7c365907 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:24.243695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340821d-7e3f-4e1a-9cfb-7ead7c365907 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:24.243695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340821d-7e3f-4e1a-9cfb-7ead7c365907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:24.249243 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:24.261297 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1340821d-7e3f-4e1a-9cfb-7ead7c365907 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:24.262528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1340821d-7e3f-4e1a-9cfb-7ead7c365907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:24.310085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceba40a-8228-43c6-98de-da99dfc4c12e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:24.330570 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 489/962] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:24.331444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340821d-7e3f-4e1a-9cfb-7ead7c365907 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:24.349833 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca4a23-f440-479f-94c5-c3ff8fd7a279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:24.350972 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca4a23-f440-479f-94c5-c3ff8fd7a279 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:24.350972 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca4a23-f440-479f-94c5-c3ff8fd7a279 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:24.351184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca4a23-f440-479f-94c5-c3ff8fd7a279 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:24.351743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca4a23-f440-479f-94c5-c3ff8fd7a279 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:24.352118 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca4a23-f440-479f-94c5-c3ff8fd7a279 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:24.353500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca4a23-f440-479f-94c5-c3ff8fd7a279 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:24.358845 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 474/963] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 3 10:31:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:24.359531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca4a23-f440-479f-94c5-c3ff8fd7a279 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:24.434726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340821d-7e3f-4e1a-9cfb-7ead7c365907 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:24.458951 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:24.463440 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca4a23-f440-479f-94c5-c3ff8fd7a279 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:24.465245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:24.470983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708e7bf-0d9e-491d-b894-0f098204e6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:24.471616 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708e7bf-0d9e-491d-b894-0f098204e6fb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:24.471897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708e7bf-0d9e-491d-b894-0f098204e6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:24.472207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708e7bf-0d9e-491d-b894-0f098204e6fb None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:24.472900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708e7bf-0d9e-491d-b894-0f098204e6fb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:24.473208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708e7bf-0d9e-491d-b894-0f098204e6fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:24.474609 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708e7bf-0d9e-491d-b894-0f098204e6fb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:24.479494 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 490/964] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 3 10:31:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:24.480291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708e7bf-0d9e-491d-b894-0f098204e6fb None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:24.504345 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf6cca-6b2a-463a-8622-161ef30eb7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:24.505003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf6cca-6b2a-463a-8622-161ef30eb7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:24.505294 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf6cca-6b2a-463a-8622-161ef30eb7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:24.505596 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf6cca-6b2a-463a-8622-161ef30eb7d1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:24.515900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf6cca-6b2a-463a-8622-161ef30eb7d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-4abf6cca-6b2a-463a-8622-161ef30eb7d1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:31:24.522019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:24.523168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:24.524013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:24.524013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:24.524013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:24.524013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:24.524013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:31:24.524013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:31:24.524013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:31:24.524013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:31:24.524013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:31:24.524013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:31:24.524013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 10:31:24.524013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:24.524013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 475/965] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Aug 3 10:31:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:24.524013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf6cca-6b2a-463a-8622-161ef30eb7d1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:24.585949 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708e7bf-0d9e-491d-b894-0f098204e6fb None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:24.621513 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a9e7d-1a1c-4b0c-9eb5-f2eef5cada20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:24.622142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a9e7d-1a1c-4b0c-9eb5-f2eef5cada20 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:24.622421 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a9e7d-1a1c-4b0c-9eb5-f2eef5cada20 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:24.622693 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a9e7d-1a1c-4b0c-9eb5-f2eef5cada20 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:24.623368 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a9e7d-1a1c-4b0c-9eb5-f2eef5cada20 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:24.623668 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a9e7d-1a1c-4b0c-9eb5-f2eef5cada20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:24.624895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf6cca-6b2a-463a-8622-161ef30eb7d1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:24.625019 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a9e7d-1a1c-4b0c-9eb5-f2eef5cada20 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:24.630628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 491/966] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 3 10:31:24 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:24.631455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a9e7d-1a1c-4b0c-9eb5-f2eef5cada20 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:24.672371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da192bb-9af6-4e52-8435-4c6ceff7027b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:24.673021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da192bb-9af6-4e52-8435-4c6ceff7027b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:24.673306 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da192bb-9af6-4e52-8435-4c6ceff7027b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:24.673623 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da192bb-9af6-4e52-8435-4c6ceff7027b None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:24.674330 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da192bb-9af6-4e52-8435-4c6ceff7027b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:24.674641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da192bb-9af6-4e52-8435-4c6ceff7027b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:24.676064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da192bb-9af6-4e52-8435-4c6ceff7027b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:24.684826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 476/967] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 3 10:31:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:24.692108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da192bb-9af6-4e52-8435-4c6ceff7027b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:24.721005 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:24.727178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:24.784053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a9e7d-1a1c-4b0c-9eb5-f2eef5cada20 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:24.815516 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da192bb-9af6-4e52-8435-4c6ceff7027b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:24.827040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8227320-3c7c-4192-8aba-6852b8df1ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:24.827671 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8227320-3c7c-4192-8aba-6852b8df1ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:24.827955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8227320-3c7c-4192-8aba-6852b8df1ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:24.828228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8227320-3c7c-4192-8aba-6852b8df1ff5 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:24.828963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8227320-3c7c-4192-8aba-6852b8df1ff5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:24.829267 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8227320-3c7c-4192-8aba-6852b8df1ff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:24.831227 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8227320-3c7c-4192-8aba-6852b8df1ff5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:24.846003 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 492/968] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Aug 3 10:31:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:24.848165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8227320-3c7c-4192-8aba-6852b8df1ff5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:24.856768 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66867774-4ea2-4373-be94-89e7cb4ecc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:24.857412 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66867774-4ea2-4373-be94-89e7cb4ecc48 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:24.858043 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66867774-4ea2-4373-be94-89e7cb4ecc48 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:24.858043 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66867774-4ea2-4373-be94-89e7cb4ecc48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:24.858706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66867774-4ea2-4373-be94-89e7cb4ecc48 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:24.860090 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66867774-4ea2-4373-be94-89e7cb4ecc48 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:24.887102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 477/969] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:31:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:31:24.981124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8227320-3c7c-4192-8aba-6852b8df1ff5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:25.008158 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:25.031686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb0cb4-8ce2-448a-b355-df95d677619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:25.032355 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb0cb4-8ce2-448a-b355-df95d677619e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:25.032640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb0cb4-8ce2-448a-b355-df95d677619e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:25.033009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb0cb4-8ce2-448a-b355-df95d677619e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:25.046853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb0cb4-8ce2-448a-b355-df95d677619e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:25.048310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb0cb4-8ce2-448a-b355-df95d677619e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:25.066744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-c5eb0cb4-8ce2-448a-b355-df95d677619e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:25.066744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:25.066744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:25.066744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:25.066744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:25.066744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:25.066744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:25.066744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:25.066744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:25.066744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:25.066744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:25.066744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:25.066744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:25.066744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:25.066744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:25.067722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:25.068634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:25.068634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:25.068634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:25.068634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:25.068634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 493/970] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:25.075117 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66867774-4ea2-4373-be94-89e7cb4ecc48 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:25.164357 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:25.190334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66867774-4ea2-4373-be94-89e7cb4ecc48 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:25.239429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b675ee-6987-400c-b22d-ab58bd043056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:25.240358 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b675ee-6987-400c-b22d-ab58bd043056 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:25.240845 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b675ee-6987-400c-b22d-ab58bd043056 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:25.241311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b675ee-6987-400c-b22d-ab58bd043056 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:25.242615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b675ee-6987-400c-b22d-ab58bd043056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:25.243146 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b675ee-6987-400c-b22d-ab58bd043056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:25.245657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b675ee-6987-400c-b22d-ab58bd043056 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:25.271565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 478/971] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:25.273808 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb0cb4-8ce2-448a-b355-df95d677619e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:25.412011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb0cb4-8ce2-448a-b355-df95d677619e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:25.450010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b32cd0-9ba3-46fe-907c-aad2800998a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:25.450010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b32cd0-9ba3-46fe-907c-aad2800998a4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:25.450010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b32cd0-9ba3-46fe-907c-aad2800998a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:25.450010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b32cd0-9ba3-46fe-907c-aad2800998a4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:25.459014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b32cd0-9ba3-46fe-907c-aad2800998a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:25.460450 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b32cd0-9ba3-46fe-907c-aad2800998a4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-e5b32cd0-9ba3-46fe-907c-aad2800998a4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:31:25.473220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:25.474289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:25.479995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:25.479995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:25.479995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:31:25.479995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:31:25.479995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:31:25.479995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:31:25.479995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:31:25.479995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:31:25.479995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 10:31:25.479995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:25.479995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 494/972] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 3 10:31:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:25.480670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b675ee-6987-400c-b22d-ab58bd043056 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:25.588407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b675ee-6987-400c-b22d-ab58bd043056 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:25.628458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3606580-3128-4e42-88ce-d1b62b879ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:25.629092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3606580-3128-4e42-88ce-d1b62b879ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:25.629370 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3606580-3128-4e42-88ce-d1b62b879ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:25.629675 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3606580-3128-4e42-88ce-d1b62b879ae5 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:25.630477 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3606580-3128-4e42-88ce-d1b62b879ae5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:25.630780 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3606580-3128-4e42-88ce-d1b62b879ae5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:25.632211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3606580-3128-4e42-88ce-d1b62b879ae5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:25.641603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:25.655669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 479/973] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Aug 3 10:31:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:25.662656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b32cd0-9ba3-46fe-907c-aad2800998a4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:25.699764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:25.780306 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b32cd0-9ba3-46fe-907c-aad2800998a4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:25.818711 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b1b66-f3d0-449e-9f0b-69bb33c18efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:25.819355 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b1b66-f3d0-449e-9f0b-69bb33c18efa None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:25.819817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b1b66-f3d0-449e-9f0b-69bb33c18efa None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:25.820101 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b1b66-f3d0-449e-9f0b-69bb33c18efa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:25.896586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b1b66-f3d0-449e-9f0b-69bb33c18efa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:25.897990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b1b66-f3d0-449e-9f0b-69bb33c18efa None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:25.932645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:26.013551 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 495/974] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:31:25 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 10:31:26.016474 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3606580-3128-4e42-88ce-d1b62b879ae5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:26.121297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3606580-3128-4e42-88ce-d1b62b879ae5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:26.156878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de514f2-582e-4363-9eda-0679f0c4ff77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:26.157450 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de514f2-582e-4363-9eda-0679f0c4ff77 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:26.157796 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de514f2-582e-4363-9eda-0679f0c4ff77 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:26.158078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de514f2-582e-4363-9eda-0679f0c4ff77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:26.168657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de514f2-582e-4363-9eda-0679f0c4ff77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:26.170193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de514f2-582e-4363-9eda-0679f0c4ff77 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:26.185479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-6de514f2-582e-4363-9eda-0679f0c4ff77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:26.185479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:26.185479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:26.185479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:26.185479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:26.185479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:26.185479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:26.185479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:26.185479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:26.185479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:26.185479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:26.185479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:26.185479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:26.185479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:26.185479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:26.186452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:26.187353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:26.187353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:26.187353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:26.187353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:26.191391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 480/975] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:26.193245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b1b66-f3d0-449e-9f0b-69bb33c18efa None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:26.303868 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b1b66-f3d0-449e-9f0b-69bb33c18efa None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:26.332921 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:26.340054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570ec86-e0c9-42bd-bfba-e2e6f346121d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:26.340684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570ec86-e0c9-42bd-bfba-e2e6f346121d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:26.340969 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570ec86-e0c9-42bd-bfba-e2e6f346121d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:26.341249 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570ec86-e0c9-42bd-bfba-e2e6f346121d None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:26.341981 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570ec86-e0c9-42bd-bfba-e2e6f346121d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:26.342288 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570ec86-e0c9-42bd-bfba-e2e6f346121d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:26.343650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570ec86-e0c9-42bd-bfba-e2e6f346121d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:26.359903 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 496/976] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:26.362104 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de514f2-582e-4363-9eda-0679f0c4ff77 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:26.412429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:26.466244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de514f2-582e-4363-9eda-0679f0c4ff77 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:26.500278 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51ada6-be92-48ef-81a5-68c9346c770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:26.500934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51ada6-be92-48ef-81a5-68c9346c770c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:26.501339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51ada6-be92-48ef-81a5-68c9346c770c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:26.502167 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51ada6-be92-48ef-81a5-68c9346c770c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:26.508904 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51ada6-be92-48ef-81a5-68c9346c770c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:26.510419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51ada6-be92-48ef-81a5-68c9346c770c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:26.522658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-2c51ada6-be92-48ef-81a5-68c9346c770c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:26.522658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:26.522658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:26.522658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:26.522658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:26.522658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:26.522658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:26.522658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:26.522658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:26.522658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:26.522658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:26.522658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:26.522658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:31:26.522658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:31:26.522658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:26.523597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:26.530100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:31:26.530100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:31:26.530100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:31:26.530100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:31:26.530100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:31:26.530100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:31:26.530100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:26.530100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:26.530100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 481/977] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:26.532181 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570ec86-e0c9-42bd-bfba-e2e6f346121d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:26.637797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570ec86-e0c9-42bd-bfba-e2e6f346121d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:26.675415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06671d-d8cb-49ff-a235-b7551c6dd863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:26.675963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06671d-d8cb-49ff-a235-b7551c6dd863 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:26.676247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06671d-d8cb-49ff-a235-b7551c6dd863 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:26.676523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06671d-d8cb-49ff-a235-b7551c6dd863 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:26.677281 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06671d-d8cb-49ff-a235-b7551c6dd863 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:26.677625 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06671d-d8cb-49ff-a235-b7551c6dd863 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:26.679043 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06671d-d8cb-49ff-a235-b7551c6dd863 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:26.694248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 497/978] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:31:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:26.695883 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e1e18-b215-45f5-8774-d863fbb0a9e7 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:26.696176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e1e18-b215-45f5-8774-d863fbb0a9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:26.696454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e1e18-b215-45f5-8774-d863fbb0a9e7 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:26.698092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 482/979] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:26.700554 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06671d-d8cb-49ff-a235-b7551c6dd863 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:26.702953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542d5cc-82df-4cf8-bdf3-15608dbfbb41 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:26.703458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542d5cc-82df-4cf8-bdf3-15608dbfbb41 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:26.703965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542d5cc-82df-4cf8-bdf3-15608dbfbb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:26.723138 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8542d5cc-82df-4cf8-bdf3-15608dbfbb41 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:26.724667 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8542d5cc-82df-4cf8-bdf3-15608dbfbb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:26.755631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:26.799837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 483/980] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:26.811840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06671d-d8cb-49ff-a235-b7551c6dd863 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:26.821685 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542d5cc-82df-4cf8-bdf3-15608dbfbb41 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:26.851282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61de5b-ca7f-49a5-b0d9-5a1a7d7aadfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:26.852077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61de5b-ca7f-49a5-b0d9-5a1a7d7aadfe None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:26.852482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61de5b-ca7f-49a5-b0d9-5a1a7d7aadfe None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:26.852764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61de5b-ca7f-49a5-b0d9-5a1a7d7aadfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:26.853609 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61de5b-ca7f-49a5-b0d9-5a1a7d7aadfe None admin] RBAC: Adding query filter params (role.id=485bbc11b43a4e36b7ccb740582ec8d2, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=035ae78ef3cb43e985e06cc3fa971c8b) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:26.853925 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61de5b-ca7f-49a5-b0d9-5a1a7d7aadfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:26.855601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61de5b-ca7f-49a5-b0d9-5a1a7d7aadfe None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:26.873514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 498/981] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:31:26 2022] GET /identity/v3/role_assignments?user.id=035ae78ef3cb43e985e06cc3fa971c8b&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=485bbc11b43a4e36b7ccb740582ec8d2 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:26.873770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:26.943006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542d5cc-82df-4cf8-bdf3-15608dbfbb41 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:26.983220 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654d57d-07e4-4853-9af5-977e0a358bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:26.983875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654d57d-07e4-4853-9af5-977e0a358bcd None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:26.984159 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654d57d-07e4-4853-9af5-977e0a358bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:26.984434 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654d57d-07e4-4853-9af5-977e0a358bcd None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:26.985163 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654d57d-07e4-4853-9af5-977e0a358bcd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:26.986754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654d57d-07e4-4853-9af5-977e0a358bcd None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:27.002847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59809991-c483-4cc3-8fbc-af31aead0ecc None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:27.003237 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59809991-c483-4cc3-8fbc-af31aead0ecc None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:27.003452 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59809991-c483-4cc3-8fbc-af31aead0ecc None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:27.004153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 484/982] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 3 10:31:26 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:31:27.004800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 499/983] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:31:27.010801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88019ae-eb65-4734-b79e-5d50f243bbec None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:27.011371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88019ae-eb65-4734-b79e-5d50f243bbec None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:27.011791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88019ae-eb65-4734-b79e-5d50f243bbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:27.032073 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a88019ae-eb65-4734-b79e-5d50f243bbec None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:27.033906 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a88019ae-eb65-4734-b79e-5d50f243bbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:27.121108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 485/984] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:27.143375 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59809991-c483-4cc3-8fbc-af31aead0ecc None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:27.267759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59809991-c483-4cc3-8fbc-af31aead0ecc None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:27.283505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:27.306063 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa52e01-90ce-4204-b9ac-a51910cc5294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:27.306768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa52e01-90ce-4204-b9ac-a51910cc5294 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:27.307055 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa52e01-90ce-4204-b9ac-a51910cc5294 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:27.307334 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa52e01-90ce-4204-b9ac-a51910cc5294 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:27.307972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa52e01-90ce-4204-b9ac-a51910cc5294 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:27.309410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa52e01-90ce-4204-b9ac-a51910cc5294 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-2fa52e01-90ce-4204-b9ac-a51910cc5294 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:27.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:27.320459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:27.321398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:27.321398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:27.321398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:27.321398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:27.321398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:27.321398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:27.321398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:27.321398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:27.321398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:27.321398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:27.321398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:27.321398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:27.321398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 10:31:27.321398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:27.322753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 500/985] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 3 10:31:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:27.327079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88019ae-eb65-4734-b79e-5d50f243bbec None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:27.353569 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:27.432645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88019ae-eb65-4734-b79e-5d50f243bbec None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:27.468898 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05670f54-e7c7-4080-bb88-5243a0fdd6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:27.469549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05670f54-e7c7-4080-bb88-5243a0fdd6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:27.469834 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05670f54-e7c7-4080-bb88-5243a0fdd6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:27.470105 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05670f54-e7c7-4080-bb88-5243a0fdd6c5 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:27.470813 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05670f54-e7c7-4080-bb88-5243a0fdd6c5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:27.471115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05670f54-e7c7-4080-bb88-5243a0fdd6c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:27.472470 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05670f54-e7c7-4080-bb88-5243a0fdd6c5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:27.481443 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 486/986] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 3 10:31:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:27.483681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa52e01-90ce-4204-b9ac-a51910cc5294 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:27.590356 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa52e01-90ce-4204-b9ac-a51910cc5294 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:27.625479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e806b-b6c9-4634-a3fb-f811a6eaa92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:27.626122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e806b-b6c9-4634-a3fb-f811a6eaa92c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:27.626404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e806b-b6c9-4634-a3fb-f811a6eaa92c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:27.627206 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e806b-b6c9-4634-a3fb-f811a6eaa92c None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:27.627904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e806b-b6c9-4634-a3fb-f811a6eaa92c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:27.628211 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e806b-b6c9-4634-a3fb-f811a6eaa92c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:27.629605 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e806b-b6c9-4634-a3fb-f811a6eaa92c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:27.639617 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 501/987] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 3 10:31:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:27.641205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05670f54-e7c7-4080-bb88-5243a0fdd6c5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:27.739184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:27.752400 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05670f54-e7c7-4080-bb88-5243a0fdd6c5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:27.790555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5390ea8-1fc4-466c-be5d-4dbae51af716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:27.791181 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5390ea8-1fc4-466c-be5d-4dbae51af716 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:27.791463 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5390ea8-1fc4-466c-be5d-4dbae51af716 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:27.791740 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5390ea8-1fc4-466c-be5d-4dbae51af716 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:27.792509 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5390ea8-1fc4-466c-be5d-4dbae51af716 None admin] RBAC: Adding query filter params (interface=admin, service_id=e9f64f52b5c14b94928b300afd795411, region_id=RegionOne) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:27.792819 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5390ea8-1fc4-466c-be5d-4dbae51af716 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:27.794209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5390ea8-1fc4-466c-be5d-4dbae51af716 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:27.804513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 487/988] 10.222.0.47 () {56 vars in 1249 bytes} [Wed Aug 3 10:31:27 2022] GET /identity/v3/endpoints?service_id=e9f64f52b5c14b94928b300afd795411&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:27.806208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e806b-b6c9-4634-a3fb-f811a6eaa92c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:27.855034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:27.919206 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e806b-b6c9-4634-a3fb-f811a6eaa92c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:27.953091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00fe33-3092-406f-a585-7de6cb6a67b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:27.953737 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00fe33-3092-406f-a585-7de6cb6a67b1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:27.954013 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00fe33-3092-406f-a585-7de6cb6a67b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:27.954283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00fe33-3092-406f-a585-7de6cb6a67b1 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:27.955496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00fe33-3092-406f-a585-7de6cb6a67b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:27.956844 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00fe33-3092-406f-a585-7de6cb6a67b1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:27.963401 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 502/989] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 3 10:31:27 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:31:28.962859 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ee662-e34a-4c23-a15d-e2a31404ae2c None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:28.963416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ee662-e34a-4c23-a15d-e2a31404ae2c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:28.963416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ee662-e34a-4c23-a15d-e2a31404ae2c None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:28.965025 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 488/990] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:28.970990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3beaa7-2094-4b39-b516-037e3e5cf0d7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:28.971446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3beaa7-2094-4b39-b516-037e3e5cf0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:28.971531 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3beaa7-2094-4b39-b516-037e3e5cf0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:28.990037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ab3beaa7-2094-4b39-b516-037e3e5cf0d7 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:28.991239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ab3beaa7-2094-4b39-b516-037e3e5cf0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:29.059764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 503/991] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:28 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:29.073600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a905063-dc21-4d16-b735-45fba03d0a63 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:29.073881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a905063-dc21-4d16-b735-45fba03d0a63 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:29.074162 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a905063-dc21-4d16-b735-45fba03d0a63 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:29.075644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 489/992] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:29.080614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024d7eb-ac20-47db-8f8c-9c758fbcb756 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:29.080771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a905063-dc21-4d16-b735-45fba03d0a63 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:29.080953 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024d7eb-ac20-47db-8f8c-9c758fbcb756 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:29.081178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024d7eb-ac20-47db-8f8c-9c758fbcb756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:29.102060 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1024d7eb-ac20-47db-8f8c-9c758fbcb756 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:29.103252 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1024d7eb-ac20-47db-8f8c-9c758fbcb756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:29.136984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:29.178144 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 504/993] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:29.197379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a905063-dc21-4d16-b735-45fba03d0a63 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:29.201259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024d7eb-ac20-47db-8f8c-9c758fbcb756 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:29.235580 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d0c38-f604-41c7-a6be-e1add2f2de98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:29.236206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d0c38-f604-41c7-a6be-e1add2f2de98 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:29.236490 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d0c38-f604-41c7-a6be-e1add2f2de98 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:29.236776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d0c38-f604-41c7-a6be-e1add2f2de98 None admin] PATH_INFO: `/v3/services/image` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:29.237413 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d0c38-f604-41c7-a6be-e1add2f2de98 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:29.238807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d0c38-f604-41c7-a6be-e1add2f2de98 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-b40d0c38-f604-41c7-a6be-e1add2f2de98 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:29.247902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:29.248897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:29.249760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:29.249760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:29.249760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:29.249760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:29.249760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:29.249760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:29.249760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:29.249760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:29.249760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:29.249760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:29.249760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:29.249760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:29.249760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 10:31:29.249760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:29.254154 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 490/994] 10.222.0.47 () {56 vars in 1105 bytes} [Wed Aug 3 10:31:29 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:29.257082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d0c38-f604-41c7-a6be-e1add2f2de98 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:29.281349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:29.323343 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024d7eb-ac20-47db-8f8c-9c758fbcb756 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:29.360863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80993e66-e7e8-4cea-8267-518437ed8100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:29.361536 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80993e66-e7e8-4cea-8267-518437ed8100 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:29.361831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80993e66-e7e8-4cea-8267-518437ed8100 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:29.362093 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80993e66-e7e8-4cea-8267-518437ed8100 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:29.362713 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80993e66-e7e8-4cea-8267-518437ed8100 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:29.364077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80993e66-e7e8-4cea-8267-518437ed8100 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:29.369565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d0c38-f604-41c7-a6be-e1add2f2de98 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-80993e66-e7e8-4cea-8267-518437ed8100 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:29.373012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:29.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:29.378682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:29.378682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:29.378682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:29.378682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:29.378682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:29.378682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:29.378682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:29.378682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:29.378682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:29.378682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:29.378682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:29.378682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:29.378682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 10:31:29.378682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:29.378682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 505/995] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:29.379987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80993e66-e7e8-4cea-8267-518437ed8100 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:29.408908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2856ea9-5579-4f3b-a5ae-547ccd97ea56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:29.409598 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2856ea9-5579-4f3b-a5ae-547ccd97ea56 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:29.409882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2856ea9-5579-4f3b-a5ae-547ccd97ea56 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:29.410157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2856ea9-5579-4f3b-a5ae-547ccd97ea56 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:29.410845 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2856ea9-5579-4f3b-a5ae-547ccd97ea56 None admin] RBAC: Adding query filter params (name=image) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:29.411153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2856ea9-5579-4f3b-a5ae-547ccd97ea56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:29.412532 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2856ea9-5579-4f3b-a5ae-547ccd97ea56 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:29.412726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:29.417805 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 491/996] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:29.423566 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2856ea9-5579-4f3b-a5ae-547ccd97ea56 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:29.494371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80993e66-e7e8-4cea-8267-518437ed8100 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:29.509702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:29.530956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2856ea9-5579-4f3b-a5ae-547ccd97ea56 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:29.534568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9feca36-0180-47c1-8734-34dcc6604db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:29.534568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9feca36-0180-47c1-8734-34dcc6604db3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:29.534800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9feca36-0180-47c1-8734-34dcc6604db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:29.535061 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9feca36-0180-47c1-8734-34dcc6604db3 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:29.535741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9feca36-0180-47c1-8734-34dcc6604db3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:29.536041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9feca36-0180-47c1-8734-34dcc6604db3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:29.537399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9feca36-0180-47c1-8734-34dcc6604db3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:29.546878 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 506/997] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 3 10:31:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:29.547924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9feca36-0180-47c1-8734-34dcc6604db3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:29.573577 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25239f-8394-4a2c-abc9-743885123cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:29.574249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25239f-8394-4a2c-abc9-743885123cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:29.574532 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25239f-8394-4a2c-abc9-743885123cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:29.574809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25239f-8394-4a2c-abc9-743885123cd0 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:29.575493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25239f-8394-4a2c-abc9-743885123cd0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:29.575797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25239f-8394-4a2c-abc9-743885123cd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:29.577169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25239f-8394-4a2c-abc9-743885123cd0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:29.589750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 492/998] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:29.665686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:29.674098 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9feca36-0180-47c1-8734-34dcc6604db3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:29.716034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfca53-1793-435b-8316-e2bf409b6aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:29.716668 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfca53-1793-435b-8316-e2bf409b6aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:29.716947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfca53-1793-435b-8316-e2bf409b6aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:29.717221 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfca53-1793-435b-8316-e2bf409b6aa2 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:29.717948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfca53-1793-435b-8316-e2bf409b6aa2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:29.718253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfca53-1793-435b-8316-e2bf409b6aa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:29.719642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfca53-1793-435b-8316-e2bf409b6aa2 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:29.725768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 507/999] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 3 10:31:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:29.731756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25239f-8394-4a2c-abc9-743885123cd0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:29.845082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25239f-8394-4a2c-abc9-743885123cd0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:29.879204 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f762043-7e6e-4695-b30f-bbdf25161137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:29.879832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f762043-7e6e-4695-b30f-bbdf25161137 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:29.880117 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f762043-7e6e-4695-b30f-bbdf25161137 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:29.880397 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f762043-7e6e-4695-b30f-bbdf25161137 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:29.881159 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f762043-7e6e-4695-b30f-bbdf25161137 None admin] RBAC: Adding query filter params (interface=public, service_id=ba51259cbd0247e2b78adfd3b1dafe4b, region_id=RegionOne) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:29.881491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f762043-7e6e-4695-b30f-bbdf25161137 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:29.882869 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f762043-7e6e-4695-b30f-bbdf25161137 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:29.894289 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 493/1000] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 3 10:31:29 2022] GET /identity/v3/endpoints?service_id=ba51259cbd0247e2b78adfd3b1dafe4b&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:29.895521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfca53-1793-435b-8316-e2bf409b6aa2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:29.997827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfca53-1793-435b-8316-e2bf409b6aa2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:30.031931 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ebb6c-9bbd-4c97-9435-5b64ed44c127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:30.032175 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afc796-fab8-46f9-a253-24862fb4b47f None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:30.032592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afc796-fab8-46f9-a253-24862fb4b47f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:30.032699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ebb6c-9bbd-4c97-9435-5b64ed44c127 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:30.032813 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afc796-fab8-46f9-a253-24862fb4b47f None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:30.032892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ebb6c-9bbd-4c97-9435-5b64ed44c127 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:30.033870 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ebb6c-9bbd-4c97-9435-5b64ed44c127 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:30.034264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 494/1001] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:31:30.037499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ebb6c-9bbd-4c97-9435-5b64ed44c127 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:30.037499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ebb6c-9bbd-4c97-9435-5b64ed44c127 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:30.039459 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc039b9-7af4-4cc3-bc11-e8b2711140e2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:30.039746 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc039b9-7af4-4cc3-bc11-e8b2711140e2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:30.040027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc039b9-7af4-4cc3-bc11-e8b2711140e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:30.042940 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 508/1002] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 3 10:31:29 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:31:30.067154 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-fbc039b9-7af4-4cc3-bc11-e8b2711140e2 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:30.068517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fbc039b9-7af4-4cc3-bc11-e8b2711140e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:30.071256 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:30.146206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 495/1003] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:30.168402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ebb6c-9bbd-4c97-9435-5b64ed44c127 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:30.197448 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:30.280855 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ebb6c-9bbd-4c97-9435-5b64ed44c127 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:30.318427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd0bda-f92e-4a61-b182-7571cb507d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:30.321823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd0bda-f92e-4a61-b182-7571cb507d6f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:30.321823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd0bda-f92e-4a61-b182-7571cb507d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:30.321823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd0bda-f92e-4a61-b182-7571cb507d6f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:30.321823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd0bda-f92e-4a61-b182-7571cb507d6f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:30.321823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd0bda-f92e-4a61-b182-7571cb507d6f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-a1fd0bda-f92e-4a61-b182-7571cb507d6f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:30.332797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:30.334196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:30.335066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:30.335066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:30.335066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:30.335066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:30.335066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:30.335066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:30.335066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:30.335066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:30.335066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:30.335066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:30.335066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:30.335066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:30.335066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 10:31:30.335066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:30.335824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 509/1004] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 3 10:31:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:30.341945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc039b9-7af4-4cc3-bc11-e8b2711140e2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:30.446051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc039b9-7af4-4cc3-bc11-e8b2711140e2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:30.479871 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec625c73-f2c2-4e2f-a5da-bfd6c692a458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:30.480506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec625c73-f2c2-4e2f-a5da-bfd6c692a458 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:30.480789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec625c73-f2c2-4e2f-a5da-bfd6c692a458 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:30.481063 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec625c73-f2c2-4e2f-a5da-bfd6c692a458 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:30.481778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec625c73-f2c2-4e2f-a5da-bfd6c692a458 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:30.482085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec625c73-f2c2-4e2f-a5da-bfd6c692a458 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:30.483444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec625c73-f2c2-4e2f-a5da-bfd6c692a458 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:30.493497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 496/1005] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 3 10:31:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:30.495068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd0bda-f92e-4a61-b182-7571cb507d6f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:30.585399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:30.602624 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd0bda-f92e-4a61-b182-7571cb507d6f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:30.637012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52feb05-1191-4eed-8dec-13da1425fe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:30.637674 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52feb05-1191-4eed-8dec-13da1425fe6e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:30.637961 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52feb05-1191-4eed-8dec-13da1425fe6e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:30.638236 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52feb05-1191-4eed-8dec-13da1425fe6e None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:30.638921 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52feb05-1191-4eed-8dec-13da1425fe6e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:30.639223 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52feb05-1191-4eed-8dec-13da1425fe6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:30.640579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52feb05-1191-4eed-8dec-13da1425fe6e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:30.651105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 510/1006] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 3 10:31:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:30.656637 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec625c73-f2c2-4e2f-a5da-bfd6c692a458 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:30.660126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:30.766959 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec625c73-f2c2-4e2f-a5da-bfd6c692a458 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:30.804020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b50368-a2da-435b-a315-cb114bd97e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:30.804692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b50368-a2da-435b-a315-cb114bd97e97 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:30.804931 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b50368-a2da-435b-a315-cb114bd97e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:30.805320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b50368-a2da-435b-a315-cb114bd97e97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:30.805974 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b50368-a2da-435b-a315-cb114bd97e97 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:30.807374 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b50368-a2da-435b-a315-cb114bd97e97 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:30.825318 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 497/1007] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:31:30 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:31:31.630716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56089587-3d82-42ee-9420-24550aaf635a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:31.631263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56089587-3d82-42ee-9420-24550aaf635a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:31.631263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56089587-3d82-42ee-9420-24550aaf635a None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:31.632752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 511/1008] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:31.637858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f995fc-c7bf-416c-a3c6-3bef869dc3f5 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:31.638273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f995fc-c7bf-416c-a3c6-3bef869dc3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:31.638415 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f995fc-c7bf-416c-a3c6-3bef869dc3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:31.663381 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-22f995fc-c7bf-416c-a3c6-3bef869dc3f5 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:31.664769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-22f995fc-c7bf-416c-a3c6-3bef869dc3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:31.743440 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:31.766742 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 498/1009] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:31 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:31.795438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56089587-3d82-42ee-9420-24550aaf635a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:31.931019 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56089587-3d82-42ee-9420-24550aaf635a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:31.990217 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41099c20-5573-423a-a068-af6de8dd7f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:31.991283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41099c20-5573-423a-a068-af6de8dd7f85 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:31.991618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41099c20-5573-423a-a068-af6de8dd7f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:31.992110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41099c20-5573-423a-a068-af6de8dd7f85 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:31.993416 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41099c20-5573-423a-a068-af6de8dd7f85 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:31.996006 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41099c20-5573-423a-a068-af6de8dd7f85 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:32.029491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 512/1010] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 3 10:31:31 2022] POST /identity/v3/services => generated 242 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:31:32.144630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2f7a6-e5be-4da6-828d-bd19e7a2c80a None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:32.144998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2f7a6-e5be-4da6-828d-bd19e7a2c80a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:32.145182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2f7a6-e5be-4da6-828d-bd19e7a2c80a None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:32.146857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 499/1011] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:32.152646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866c5ea-1d75-4476-b741-b84ab18c64b9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:32.152967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866c5ea-1d75-4476-b741-b84ab18c64b9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:32.153203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866c5ea-1d75-4476-b741-b84ab18c64b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:32.187436 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d866c5ea-1d75-4476-b741-b84ab18c64b9 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:32.188634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d866c5ea-1d75-4476-b741-b84ab18c64b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:32.193200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:32.296277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 513/1012] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:32 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:32.334871 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2f7a6-e5be-4da6-828d-bd19e7a2c80a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:32.444454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2f7a6-e5be-4da6-828d-bd19e7a2c80a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:32.478818 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ec540-5eb3-437b-8770-7c137e1355b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:32.479466 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ec540-5eb3-437b-8770-7c137e1355b1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:32.479752 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ec540-5eb3-437b-8770-7c137e1355b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:32.480025 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ec540-5eb3-437b-8770-7c137e1355b1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:32.480661 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ec540-5eb3-437b-8770-7c137e1355b1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:32.482161 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ec540-5eb3-437b-8770-7c137e1355b1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-af2ec540-5eb3-437b-8770-7c137e1355b1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:32.491978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:32.493077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:32.493960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:32.493960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:32.493960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:32.493960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:32.493960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:32.493960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:32.493960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:32.493960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:32.493960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:32.493960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:32.493960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:32.493960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:32.493960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 10:31:32.493960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:32.497542 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 500/1013] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:32.499168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866c5ea-1d75-4476-b741-b84ab18c64b9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:32.602341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866c5ea-1d75-4476-b741-b84ab18c64b9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:32.636579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1417d48-efad-4a9b-8c89-934fb58ba4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:32.637277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1417d48-efad-4a9b-8c89-934fb58ba4dd None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:32.637555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1417d48-efad-4a9b-8c89-934fb58ba4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:32.637762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1417d48-efad-4a9b-8c89-934fb58ba4dd None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:32.638440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1417d48-efad-4a9b-8c89-934fb58ba4dd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:32.638741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1417d48-efad-4a9b-8c89-934fb58ba4dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:32.640087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1417d48-efad-4a9b-8c89-934fb58ba4dd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:32.642005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:32.651023 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 514/1014] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 3 10:31:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:32.656467 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ec540-5eb3-437b-8770-7c137e1355b1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:32.714106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:32.762339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ec540-5eb3-437b-8770-7c137e1355b1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:32.796236 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4db13-ec6f-479d-a280-51e1042d12e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:32.796858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4db13-ec6f-479d-a280-51e1042d12e9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:32.797136 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4db13-ec6f-479d-a280-51e1042d12e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:32.797404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4db13-ec6f-479d-a280-51e1042d12e9 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:32.798104 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4db13-ec6f-479d-a280-51e1042d12e9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:32.799015 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4db13-ec6f-479d-a280-51e1042d12e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:32.800380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4db13-ec6f-479d-a280-51e1042d12e9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:32.809509 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 501/1015] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 3 10:31:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:32.811622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1417d48-efad-4a9b-8c89-934fb58ba4dd None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:32.907378 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1f976-43e2-4eca-b881-c2d47e2b92ac None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:32.908398 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1f976-43e2-4eca-b881-c2d47e2b92ac None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:32.908734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1f976-43e2-4eca-b881-c2d47e2b92ac None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:32.910391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 502/1016] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:31:32.913774 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1417d48-efad-4a9b-8c89-934fb58ba4dd None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:32.915468 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876a3fc-2a7c-4e11-bb8d-5802e3628fb4 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:32.915758 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876a3fc-2a7c-4e11-bb8d-5802e3628fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:32.916042 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876a3fc-2a7c-4e11-bb8d-5802e3628fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:32.934194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4876a3fc-2a7c-4e11-bb8d-5802e3628fb4 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:32.935399 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4876a3fc-2a7c-4e11-bb8d-5802e3628fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:32.949329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db424f-87b1-4cd3-bc95-1684f2492564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:32.949979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db424f-87b1-4cd3-bc95-1684f2492564 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:32.950259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db424f-87b1-4cd3-bc95-1684f2492564 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:32.950529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db424f-87b1-4cd3-bc95-1684f2492564 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:32.951239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db424f-87b1-4cd3-bc95-1684f2492564 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:32.952582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db424f-87b1-4cd3-bc95-1684f2492564 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:32.985577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 515/1017] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:31:32 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:31:33.014138 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 503/1018] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:33.038276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db424f-87b1-4cd3-bc95-1684f2492564 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:33.102656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:33.155577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db424f-87b1-4cd3-bc95-1684f2492564 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:33.190431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6879271-3267-44cb-89fb-bf4a6d24569b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:33.191086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6879271-3267-44cb-89fb-bf4a6d24569b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:33.191365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6879271-3267-44cb-89fb-bf4a6d24569b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:33.191636 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6879271-3267-44cb-89fb-bf4a6d24569b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:33.192807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6879271-3267-44cb-89fb-bf4a6d24569b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:33.194269 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6879271-3267-44cb-89fb-bf4a6d24569b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:33.196070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 516/1019] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:33.203856 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876a3fc-2a7c-4e11-bb8d-5802e3628fb4 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:33.256794 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:33.335145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876a3fc-2a7c-4e11-bb8d-5802e3628fb4 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:33.369014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8a6be-611a-4493-8e8c-912729048159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:33.369673 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8a6be-611a-4493-8e8c-912729048159 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:33.369959 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8a6be-611a-4493-8e8c-912729048159 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:33.370233 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8a6be-611a-4493-8e8c-912729048159 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:33.370914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8a6be-611a-4493-8e8c-912729048159 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:33.372351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8a6be-611a-4493-8e8c-912729048159 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:33.405092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 504/1020] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 3 10:31:33 2022] POST /identity/v3/projects => generated 311 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:31:34.125080 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decdb94-0645-4710-bc81-17fc046e6732 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:34.125668 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decdb94-0645-4710-bc81-17fc046e6732 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:34.125668 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decdb94-0645-4710-bc81-17fc046e6732 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:34.127311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 517/1021] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:34.132407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b63d7-b22f-4c9d-a786-6ef92f6d60c7 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:34.132835 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b63d7-b22f-4c9d-a786-6ef92f6d60c7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:34.133038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b63d7-b22f-4c9d-a786-6ef92f6d60c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:34.151312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d37b63d7-b22f-4c9d-a786-6ef92f6d60c7 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:34.152513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d37b63d7-b22f-4c9d-a786-6ef92f6d60c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:34.226472 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 505/1022] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:34.249556 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decdb94-0645-4710-bc81-17fc046e6732 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:34.394833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decdb94-0645-4710-bc81-17fc046e6732 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:34.436322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:34.457795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b8f11-0a7e-4764-87f5-50cca33988da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:34.458468 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b8f11-0a7e-4764-87f5-50cca33988da None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:34.458924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b8f11-0a7e-4764-87f5-50cca33988da None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:34.459208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b8f11-0a7e-4764-87f5-50cca33988da None admin] PATH_INFO: `/v3/services/image` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:34.460465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b8f11-0a7e-4764-87f5-50cca33988da None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:34.462157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b8f11-0a7e-4764-87f5-50cca33988da None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-520b8f11-0a7e-4764-87f5-50cca33988da None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:34.472469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:34.473877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:34.475045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:34.475045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:34.475045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:34.475045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:34.475045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:34.475045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:34.475045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:34.475045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:34.475045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:34.475045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:34.475045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:34.475045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:34.475045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 10:31:34.475045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:34.477825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 518/1023] 10.222.0.47 () {56 vars in 1105 bytes} [Wed Aug 3 10:31:34 2022] GET /identity/v3/services/image => generated 86 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:34.479653 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b63d7-b22f-4c9d-a786-6ef92f6d60c7 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:34.508019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:34.592651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b63d7-b22f-4c9d-a786-6ef92f6d60c7 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:34.630086 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc974f8-3091-4d1f-b71b-0233a5db3720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:34.630727 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc974f8-3091-4d1f-b71b-0233a5db3720 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:34.631010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc974f8-3091-4d1f-b71b-0233a5db3720 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:34.631283 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc974f8-3091-4d1f-b71b-0233a5db3720 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:34.631970 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc974f8-3091-4d1f-b71b-0233a5db3720 None admin] RBAC: Adding query filter params (name=image) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:34.632417 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc974f8-3091-4d1f-b71b-0233a5db3720 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:34.634122 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc974f8-3091-4d1f-b71b-0233a5db3720 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:34.647651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 506/1024] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:34.652633 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b8f11-0a7e-4764-87f5-50cca33988da None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:34.790949 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b8f11-0a7e-4764-87f5-50cca33988da None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:34.849723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c3f0e-7a8f-4397-b569-a62685f1ff06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:34.850791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c3f0e-7a8f-4397-b569-a62685f1ff06 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:34.851289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c3f0e-7a8f-4397-b569-a62685f1ff06 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:34.851698 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c3f0e-7a8f-4397-b569-a62685f1ff06 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:34.852856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c3f0e-7a8f-4397-b569-a62685f1ff06 None admin] RBAC: Adding query filter params (type=image) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:34.853419 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c3f0e-7a8f-4397-b569-a62685f1ff06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:34.855919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c3f0e-7a8f-4397-b569-a62685f1ff06 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:34.869298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 519/1025] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:34.871065 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc974f8-3091-4d1f-b71b-0233a5db3720 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:34.966545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:34.980169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc974f8-3091-4d1f-b71b-0233a5db3720 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:35.019518 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a3e81-2b3a-48f2-9a80-239982338aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:35.020219 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a3e81-2b3a-48f2-9a80-239982338aad None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:35.020425 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a3e81-2b3a-48f2-9a80-239982338aad None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:35.020700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a3e81-2b3a-48f2-9a80-239982338aad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:35.021456 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a3e81-2b3a-48f2-9a80-239982338aad None admin] RBAC: Adding query filter params (interface=public, service_id=1efe2e55f23040cf9ab9ceedec1348fd, region_id=RegionOne) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:35.021789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a3e81-2b3a-48f2-9a80-239982338aad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:35.023154 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a3e81-2b3a-48f2-9a80-239982338aad None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:35.029005 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 507/1026] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 3 10:31:34 2022] GET /identity/v3/endpoints?service_id=1efe2e55f23040cf9ab9ceedec1348fd&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:35.034741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c3f0e-7a8f-4397-b569-a62685f1ff06 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:35.101252 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:35.147781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fbf12-bd29-4700-b307-2b51f17ac415 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:35.148354 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fbf12-bd29-4700-b307-2b51f17ac415 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:35.148354 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fbf12-bd29-4700-b307-2b51f17ac415 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:35.149940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 508/1027] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:31:35.155364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cc7d3-7393-40a4-8779-e6e85ae60788 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:35.155662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cc7d3-7393-40a4-8779-e6e85ae60788 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:35.155926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cc7d3-7393-40a4-8779-e6e85ae60788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:35.158339 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c3f0e-7a8f-4397-b569-a62685f1ff06 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:35.175506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e65cc7d3-7393-40a4-8779-e6e85ae60788 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:35.176741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e65cc7d3-7393-40a4-8779-e6e85ae60788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:35.197768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bb9ba-b3f5-4d79-90ec-13d2013e5627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:35.198393 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bb9ba-b3f5-4d79-90ec-13d2013e5627 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:35.198761 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bb9ba-b3f5-4d79-90ec-13d2013e5627 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:35.199010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bb9ba-b3f5-4d79-90ec-13d2013e5627 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:35.199750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20bb9ba-b3f5-4d79-90ec-13d2013e5627 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:35.201254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20bb9ba-b3f5-4d79-90ec-13d2013e5627 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:35.210653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 520/1028] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 3 10:31:35 2022] GET /identity/v3/services => generated 1957 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:31:35.268069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 509/1029] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:35.291618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bb9ba-b3f5-4d79-90ec-13d2013e5627 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:35.408569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bb9ba-b3f5-4d79-90ec-13d2013e5627 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:35.426335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:35.446111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ebf4fe-c883-43f7-9506-73ab872cbe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:35.446769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebf4fe-c883-43f7-9506-73ab872cbe3c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:35.447049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebf4fe-c883-43f7-9506-73ab872cbe3c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:35.447321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebf4fe-c883-43f7-9506-73ab872cbe3c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:35.457536 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ebf4fe-c883-43f7-9506-73ab872cbe3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-76ebf4fe-c883-43f7-9506-73ab872cbe3c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:31:35.462978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:35.463883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:35.465002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:35.465002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:35.465002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:35.465002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:35.465002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:31:35.465002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:31:35.465002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:31:35.465002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:31:35.465002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:31:35.465002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:31:35.465002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 10:31:35.465002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:35.465002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 521/1030] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Aug 3 10:31:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:35.470814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cc7d3-7393-40a4-8779-e6e85ae60788 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:35.471543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3a5d6-90b9-495f-a359-2d7a5fe7cada None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:35.471826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3a5d6-90b9-495f-a359-2d7a5fe7cada None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:35.472106 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3a5d6-90b9-495f-a359-2d7a5fe7cada None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:35.473628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 522/1031] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:35.478862 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b80e8d-58e0-4a0e-9fba-566384292af6 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:35.479146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b80e8d-58e0-4a0e-9fba-566384292af6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:35.479419 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b80e8d-58e0-4a0e-9fba-566384292af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:35.499936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d3b80e8d-58e0-4a0e-9fba-566384292af6 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:35.501137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d3b80e8d-58e0-4a0e-9fba-566384292af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:35.556075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:35.573090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 523/1032] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:35.577069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cc7d3-7393-40a4-8779-e6e85ae60788 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:35.597673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b80e8d-58e0-4a0e-9fba-566384292af6 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:35.617112 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688938a-c8fe-4265-ad98-b5b29d1db634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:35.617889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688938a-c8fe-4265-ad98-b5b29d1db634 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:35.618234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688938a-c8fe-4265-ad98-b5b29d1db634 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:35.618561 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688938a-c8fe-4265-ad98-b5b29d1db634 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:35.619282 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688938a-c8fe-4265-ad98-b5b29d1db634 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:35.619583 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688938a-c8fe-4265-ad98-b5b29d1db634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:35.620952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688938a-c8fe-4265-ad98-b5b29d1db634 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:35.630188 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 510/1033] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Aug 3 10:31:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:35.635360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688938a-c8fe-4265-ad98-b5b29d1db634 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:35.715420 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b80e8d-58e0-4a0e-9fba-566384292af6 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:35.724235 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:35.751935 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688938a-c8fe-4265-ad98-b5b29d1db634 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:35.753313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed4e58-0dbd-4f51-9d12-ec925c17c467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:35.753996 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed4e58-0dbd-4f51-9d12-ec925c17c467 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:35.754282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed4e58-0dbd-4f51-9d12-ec925c17c467 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:35.754555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed4e58-0dbd-4f51-9d12-ec925c17c467 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:35.755720 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed4e58-0dbd-4f51-9d12-ec925c17c467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:35.757145 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed4e58-0dbd-4f51-9d12-ec925c17c467 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:35.758975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 524/1034] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:35.764377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed4e58-0dbd-4f51-9d12-ec925c17c467 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:35.788521 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa448f3b-2b21-4afd-a62d-45ea1ed94fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:35.789178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa448f3b-2b21-4afd-a62d-45ea1ed94fba None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:35.789533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa448f3b-2b21-4afd-a62d-45ea1ed94fba None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:35.789816 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa448f3b-2b21-4afd-a62d-45ea1ed94fba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:35.802196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa448f3b-2b21-4afd-a62d-45ea1ed94fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:35.803649 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa448f3b-2b21-4afd-a62d-45ea1ed94fba None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:35.806303 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:35.819632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-aa448f3b-2b21-4afd-a62d-45ea1ed94fba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:35.819632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:35.819632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:35.819632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:35.819632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:35.819632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:35.819632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:35.819632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:35.819632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:35.819632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:35.819632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:35.819632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:35.819632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:35.819632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:35.819632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:35.820608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:35.822391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:35.822391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:35.822391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:35.822391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:35.822391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 511/1035] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:35.826442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa448f3b-2b21-4afd-a62d-45ea1ed94fba None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:35.878462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed4e58-0dbd-4f51-9d12-ec925c17c467 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:35.915050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044c665-8fa7-4354-9f2a-3d57b212444c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:35.915717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044c665-8fa7-4354-9f2a-3d57b212444c None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:35.916001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044c665-8fa7-4354-9f2a-3d57b212444c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:35.916273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044c665-8fa7-4354-9f2a-3d57b212444c None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:35.916963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044c665-8fa7-4354-9f2a-3d57b212444c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:35.918407 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044c665-8fa7-4354-9f2a-3d57b212444c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:35.933272 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa448f3b-2b21-4afd-a62d-45ea1ed94fba None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:35.945286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 525/1036] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:31:35 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:31:35.975021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc612078-af9f-4399-8a99-c5f4f1ecbb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:35.975789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc612078-af9f-4399-8a99-c5f4f1ecbb23 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:35.976136 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc612078-af9f-4399-8a99-c5f4f1ecbb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:35.976488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc612078-af9f-4399-8a99-c5f4f1ecbb23 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:35.977253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc612078-af9f-4399-8a99-c5f4f1ecbb23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:35.977655 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc612078-af9f-4399-8a99-c5f4f1ecbb23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:35.979115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc612078-af9f-4399-8a99-c5f4f1ecbb23 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:35.997657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 512/1037] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:35.999437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044c665-8fa7-4354-9f2a-3d57b212444c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:36.035349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:36.113793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044c665-8fa7-4354-9f2a-3d57b212444c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:36.149630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c99f6-0adf-457f-aa26-ff487be7f10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:36.150291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c99f6-0adf-457f-aa26-ff487be7f10a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:36.150566 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c99f6-0adf-457f-aa26-ff487be7f10a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:36.150838 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c99f6-0adf-457f-aa26-ff487be7f10a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:36.158710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c99f6-0adf-457f-aa26-ff487be7f10a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:36.160159 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c99f6-0adf-457f-aa26-ff487be7f10a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-f88c99f6-0adf-457f-aa26-ff487be7f10a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:31:36.172409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:36.173619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:36.174613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:36.174613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:36.174613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:31:36.174613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:31:36.174613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:31:36.174613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:31:36.174613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:31:36.174613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:31:36.174613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:31:36.174613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:36.174613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 526/1038] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 3 10:31:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:36.181332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc612078-af9f-4399-8a99-c5f4f1ecbb23 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:36.246476 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:36.291937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc612078-af9f-4399-8a99-c5f4f1ecbb23 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:36.327269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e9dac-3038-4aed-b91c-a211a51b1bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:36.327899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e9dac-3038-4aed-b91c-a211a51b1bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:36.328184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e9dac-3038-4aed-b91c-a211a51b1bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:36.328464 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e9dac-3038-4aed-b91c-a211a51b1bd0 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:36.329267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e9dac-3038-4aed-b91c-a211a51b1bd0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:36.330158 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e9dac-3038-4aed-b91c-a211a51b1bd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:36.331704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e9dac-3038-4aed-b91c-a211a51b1bd0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:36.352935 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 513/1039] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 3 10:31:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:36.358504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c99f6-0adf-457f-aa26-ff487be7f10a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:36.454386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:36.465943 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c99f6-0adf-457f-aa26-ff487be7f10a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:36.502240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745539b-4178-4473-a4f5-4144d842937c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:36.502862 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745539b-4178-4473-a4f5-4144d842937c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:36.503140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745539b-4178-4473-a4f5-4144d842937c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:36.503409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745539b-4178-4473-a4f5-4144d842937c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:36.571738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745539b-4178-4473-a4f5-4144d842937c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:36.573263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745539b-4178-4473-a4f5-4144d842937c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:36.662895 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:36.678757 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 527/1040] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:31:36 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 10:31:36.680096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e9dac-3038-4aed-b91c-a211a51b1bd0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:36.786934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e9dac-3038-4aed-b91c-a211a51b1bd0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:36.801569 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:36.823982 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0bafc1-4ce6-4865-80d2-83f2e9a66bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:36.824986 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bafc1-4ce6-4865-80d2-83f2e9a66bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:36.825419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bafc1-4ce6-4865-80d2-83f2e9a66bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:36.825908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bafc1-4ce6-4865-80d2-83f2e9a66bc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:36.836695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0bafc1-4ce6-4865-80d2-83f2e9a66bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:36.838376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0bafc1-4ce6-4865-80d2-83f2e9a66bc8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:36.853553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-db0bafc1-4ce6-4865-80d2-83f2e9a66bc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:36.853553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:36.853553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:36.853553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:36.853553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:36.853553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:36.853553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:36.853553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:36.853553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:36.853553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:36.853553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:36.853553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:36.853553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:36.853553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:36.853553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:36.854536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:36.855445 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:36.855445 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:36.855445 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:36.855445 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:36.857544 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 514/1041] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:36.861610 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745539b-4178-4473-a4f5-4144d842937c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:36.961727 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745539b-4178-4473-a4f5-4144d842937c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:36.996143 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe52e9c-ec0c-40cd-a177-6ede4aae11e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:36.996777 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe52e9c-ec0c-40cd-a177-6ede4aae11e2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:36.997058 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe52e9c-ec0c-40cd-a177-6ede4aae11e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:36.997333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe52e9c-ec0c-40cd-a177-6ede4aae11e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:36.998054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe52e9c-ec0c-40cd-a177-6ede4aae11e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:36.998358 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe52e9c-ec0c-40cd-a177-6ede4aae11e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:36.999719 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe52e9c-ec0c-40cd-a177-6ede4aae11e2 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:37.014904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 528/1042] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:37.020684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0bafc1-4ce6-4865-80d2-83f2e9a66bc8 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:37.124260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0bafc1-4ce6-4865-80d2-83f2e9a66bc8 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:37.176095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393f1f9-6ee8-4580-9784-619038a2153d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:37.177201 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393f1f9-6ee8-4580-9784-619038a2153d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:37.177738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393f1f9-6ee8-4580-9784-619038a2153d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:37.178241 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393f1f9-6ee8-4580-9784-619038a2153d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:37.189990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393f1f9-6ee8-4580-9784-619038a2153d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:37.192521 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393f1f9-6ee8-4580-9784-619038a2153d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:37.214979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-1393f1f9-6ee8-4580-9784-619038a2153d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:37.214979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:37.214979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:37.214979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:37.214979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:37.214979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:37.214979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:37.214979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:37.214979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:37.214979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:37.214979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:37.214979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:37.214979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:31:37.214979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:31:37.214979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:37.215925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:37.216817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:31:37.216817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:31:37.216817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:31:37.216817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:31:37.216817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:31:37.216817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:31:37.216817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:37.216817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:37.221841 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 515/1043] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:37.226737 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe52e9c-ec0c-40cd-a177-6ede4aae11e2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:37.258882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6736e1-d2d1-41a0-a6a0-8d05bd7ea141 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:37.259406 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6736e1-d2d1-41a0-a6a0-8d05bd7ea141 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:37.259743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6736e1-d2d1-41a0-a6a0-8d05bd7ea141 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:37.261962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 516/1044] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:31:37.267763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead23b6d-ee05-4d93-84dd-fc12a2951414 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:37.268051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead23b6d-ee05-4d93-84dd-fc12a2951414 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:37.268332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead23b6d-ee05-4d93-84dd-fc12a2951414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:37.285019 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:37.287744 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ead23b6d-ee05-4d93-84dd-fc12a2951414 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:37.288986 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ead23b6d-ee05-4d93-84dd-fc12a2951414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:37.311202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:37.342432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe52e9c-ec0c-40cd-a177-6ede4aae11e2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:37.377069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 517/1045] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:37.392373 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485a3fa-9d9d-4f3f-abb8-b2f14613e58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:37.393142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485a3fa-9d9d-4f3f-abb8-b2f14613e58a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:37.393558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485a3fa-9d9d-4f3f-abb8-b2f14613e58a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:37.394031 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485a3fa-9d9d-4f3f-abb8-b2f14613e58a None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:37.394808 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485a3fa-9d9d-4f3f-abb8-b2f14613e58a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:37.395238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485a3fa-9d9d-4f3f-abb8-b2f14613e58a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:37.396767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485a3fa-9d9d-4f3f-abb8-b2f14613e58a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:37.400095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead23b6d-ee05-4d93-84dd-fc12a2951414 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:37.412948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 529/1046] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:31:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:37.419719 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485a3fa-9d9d-4f3f-abb8-b2f14613e58a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:37.521561 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead23b6d-ee05-4d93-84dd-fc12a2951414 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:37.536123 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485a3fa-9d9d-4f3f-abb8-b2f14613e58a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:37.548256 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:37.560982 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4adfa-dbe8-4a03-b37d-8b8ab99eb453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:37.561865 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4adfa-dbe8-4a03-b37d-8b8ab99eb453 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:37.562243 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4adfa-dbe8-4a03-b37d-8b8ab99eb453 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:37.562611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4adfa-dbe8-4a03-b37d-8b8ab99eb453 None admin] PATH_INFO: `/v3/services/image` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:37.563354 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4adfa-dbe8-4a03-b37d-8b8ab99eb453 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:37.564883 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4adfa-dbe8-4a03-b37d-8b8ab99eb453 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-93a4adfa-dbe8-4a03-b37d-8b8ab99eb453 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:37.574764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:37.576038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:37.577077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:37.577077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:37.577077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:37.577077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:37.577077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:37.577077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:37.577077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:37.577077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:37.577077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:37.577077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:37.577077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:31:37.577077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:31:37.577077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 10:31:37.577077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:37.578868 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892d0c2-7d4e-48bf-9ddc-5a479ab4ab99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:37.579330 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 518/1047] 10.222.0.47 () {56 vars in 1105 bytes} [Wed Aug 3 10:31:37 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:37.585340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892d0c2-7d4e-48bf-9ddc-5a479ab4ab99 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:37.585641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4adfa-dbe8-4a03-b37d-8b8ab99eb453 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:37.585803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892d0c2-7d4e-48bf-9ddc-5a479ab4ab99 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:37.585943 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892d0c2-7d4e-48bf-9ddc-5a479ab4ab99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:37.586795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892d0c2-7d4e-48bf-9ddc-5a479ab4ab99 None admin] RBAC: Adding query filter params (role.id=485bbc11b43a4e36b7ccb740582ec8d2, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=e66f25cc329f418eba6bd637bf3e8a3a) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:37.587107 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892d0c2-7d4e-48bf-9ddc-5a479ab4ab99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:37.588583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892d0c2-7d4e-48bf-9ddc-5a479ab4ab99 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:37.595690 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 530/1048] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:31:37 2022] GET /identity/v3/role_assignments?user.id=e66f25cc329f418eba6bd637bf3e8a3a&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=485bbc11b43a4e36b7ccb740582ec8d2 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:37.614648 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:37.714562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4adfa-dbe8-4a03-b37d-8b8ab99eb453 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:37.752800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c94728-f346-484e-bb4f-b261cfc71ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:37.753563 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c94728-f346-484e-bb4f-b261cfc71ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:37.753909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c94728-f346-484e-bb4f-b261cfc71ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:37.754248 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c94728-f346-484e-bb4f-b261cfc71ec2 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:37.755011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c94728-f346-484e-bb4f-b261cfc71ec2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:37.755377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c94728-f346-484e-bb4f-b261cfc71ec2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:37.756805 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c94728-f346-484e-bb4f-b261cfc71ec2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:37.762820 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 519/1049] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:37.768618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892d0c2-7d4e-48bf-9ddc-5a479ab4ab99 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:37.872279 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892d0c2-7d4e-48bf-9ddc-5a479ab4ab99 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:37.906392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6a0c2-24b9-4925-beba-93cce31772fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:37.907039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6a0c2-24b9-4925-beba-93cce31772fa None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:37.907279 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6a0c2-24b9-4925-beba-93cce31772fa None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:37.907707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6a0c2-24b9-4925-beba-93cce31772fa None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:37.908479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6a0c2-24b9-4925-beba-93cce31772fa None admin] RBAC: Adding query filter params (type=image) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:37.908790 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6a0c2-24b9-4925-beba-93cce31772fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:37.910450 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6a0c2-24b9-4925-beba-93cce31772fa None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:37.917662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 531/1050] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:37.921843 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c94728-f346-484e-bb4f-b261cfc71ec2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:38.008875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:38.026757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c94728-f346-484e-bb4f-b261cfc71ec2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:38.054897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5410be-ec19-49ab-b664-7e5587e0fe62 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:38.055203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5410be-ec19-49ab-b664-7e5587e0fe62 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:38.057510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5410be-ec19-49ab-b664-7e5587e0fe62 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:38.057510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 532/1051] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:31:38.062458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf90a0-5ff5-4990-aa1e-6ca96a0e9f3c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:38.062772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf90a0-5ff5-4990-aa1e-6ca96a0e9f3c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:38.063045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf90a0-5ff5-4990-aa1e-6ca96a0e9f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:38.064500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7daca-c48a-470d-bc7d-199e09c02bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:38.065128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7daca-c48a-470d-bc7d-199e09c02bf5 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:38.065407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7daca-c48a-470d-bc7d-199e09c02bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:38.065749 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7daca-c48a-470d-bc7d-199e09c02bf5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:38.066479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7daca-c48a-470d-bc7d-199e09c02bf5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:38.067885 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7daca-c48a-470d-bc7d-199e09c02bf5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:38.085379 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-fdcf90a0-5ff5-4990-aa1e-6ca96a0e9f3c None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:38.086612 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fdcf90a0-5ff5-4990-aa1e-6ca96a0e9f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:38.093494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 520/1052] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:31:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:31:38.130473 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:38.181620 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 533/1053] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:38.205738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7daca-c48a-470d-bc7d-199e09c02bf5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:38.318975 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7daca-c48a-470d-bc7d-199e09c02bf5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:38.353154 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299bc00-a9a4-4c35-b0fb-13fb54aecd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:38.353828 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299bc00-a9a4-4c35-b0fb-13fb54aecd6e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:38.354110 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299bc00-a9a4-4c35-b0fb-13fb54aecd6e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:38.354379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299bc00-a9a4-4c35-b0fb-13fb54aecd6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:38.362993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:38.366621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1299bc00-a9a4-4c35-b0fb-13fb54aecd6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-1299bc00-a9a4-4c35-b0fb-13fb54aecd6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:31:38.372092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:38.373232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:38.374312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:38.374312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:38.374312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:38.374312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:38.374312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:31:38.374312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:31:38.374312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:31:38.374312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:31:38.374312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:31:38.374312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:31:38.374312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:31:38.374312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:38.381499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 521/1054] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:31:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:38.383814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf90a0-5ff5-4990-aa1e-6ca96a0e9f3c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:38.485096 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf90a0-5ff5-4990-aa1e-6ca96a0e9f3c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:38.520209 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0503e10-ccdc-459a-b1b1-2ba284a46719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:38.520913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0503e10-ccdc-459a-b1b1-2ba284a46719 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:38.521107 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0503e10-ccdc-459a-b1b1-2ba284a46719 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:38.521378 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0503e10-ccdc-459a-b1b1-2ba284a46719 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:38.522122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0503e10-ccdc-459a-b1b1-2ba284a46719 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:38.522423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0503e10-ccdc-459a-b1b1-2ba284a46719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:38.523780 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0503e10-ccdc-459a-b1b1-2ba284a46719 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:38.536980 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 534/1055] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:31:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:38.537957 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299bc00-a9a4-4c35-b0fb-13fb54aecd6e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:38.642494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299bc00-a9a4-4c35-b0fb-13fb54aecd6e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:38.677127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9cea3-4624-46ce-ae1a-608d3e6ce6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:38.677804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9cea3-4624-46ce-ae1a-608d3e6ce6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:38.678084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9cea3-4624-46ce-ae1a-608d3e6ce6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:38.678356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9cea3-4624-46ce-ae1a-608d3e6ce6c5 None admin] PATH_INFO: `/v3/users/ac8bf30fd2ea48d5b41389a2bd0a755d` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:38.696336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9cea3-4624-46ce-ae1a-608d3e6ce6c5 None admin] RBAC: Authorizing `identity:get_user(user_id=ac8bf30fd2ea48d5b41389a2bd0a755d)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:38.697817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9cea3-4624-46ce-ae1a-608d3e6ce6c5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:38.704413 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 522/1056] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:31:38 2022] GET /identity/v3/users/ac8bf30fd2ea48d5b41389a2bd0a755d => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:38.705397 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0503e10-ccdc-459a-b1b1-2ba284a46719 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:38.735355 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:38.809164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0503e10-ccdc-459a-b1b1-2ba284a46719 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:38.842482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84655cd-db1c-45c1-bda0-d7c21e229ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:38.843099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84655cd-db1c-45c1-bda0-d7c21e229ceb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:38.843377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84655cd-db1c-45c1-bda0-d7c21e229ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:38.843650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84655cd-db1c-45c1-bda0-d7c21e229ceb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:38.912021 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84655cd-db1c-45c1-bda0-d7c21e229ceb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:38.913350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84655cd-db1c-45c1-bda0-d7c21e229ceb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:38.939249 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:39.016243 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 535/1057] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:31:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 10:31:39.018310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9cea3-4624-46ce-ae1a-608d3e6ce6c5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:39.077768 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:39.121670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9cea3-4624-46ce-ae1a-608d3e6ce6c5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:39.155817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153f20e-74b0-48db-a3f0-c2b4d8d92f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:39.156460 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153f20e-74b0-48db-a3f0-c2b4d8d92f6f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:39.156737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153f20e-74b0-48db-a3f0-c2b4d8d92f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:39.157008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153f20e-74b0-48db-a3f0-c2b4d8d92f6f None admin] PATH_INFO: `/v3/projects/8a2b0c845b564700904b853a303f55e7` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:39.158185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153f20e-74b0-48db-a3f0-c2b4d8d92f6f None admin] RBAC: Authorizing `identity:get_project(project_id=8a2b0c845b564700904b853a303f55e7)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:39.159600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153f20e-74b0-48db-a3f0-c2b4d8d92f6f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:39.166253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 523/1058] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:31:39 2022] GET /identity/v3/projects/8a2b0c845b564700904b853a303f55e7 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:39.167481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84655cd-db1c-45c1-bda0-d7c21e229ceb None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:39.279825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84655cd-db1c-45c1-bda0-d7c21e229ceb None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:39.306802 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:39.317459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64c5f4-1866-4cf7-b6f5-b72a838b6028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:39.318106 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64c5f4-1866-4cf7-b6f5-b72a838b6028 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:39.318387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64c5f4-1866-4cf7-b6f5-b72a838b6028 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:39.318661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64c5f4-1866-4cf7-b6f5-b72a838b6028 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:39.319465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64c5f4-1866-4cf7-b6f5-b72a838b6028 None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=8a2b0c845b564700904b853a303f55e7, user.id=ac8bf30fd2ea48d5b41389a2bd0a755d) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:39.319774 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64c5f4-1866-4cf7-b6f5-b72a838b6028 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:39.321197 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64c5f4-1866-4cf7-b6f5-b72a838b6028 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:39.333506 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 536/1059] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:31:39 2022] GET /identity/v3/role_assignments?user.id=ac8bf30fd2ea48d5b41389a2bd0a755d&scope.project.id=8a2b0c845b564700904b853a303f55e7&role.id=01714797cab143e1a7a150c068fa9315 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:39.624895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc967-d754-4a91-b15c-209fc78f969f None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:39.625436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc967-d754-4a91-b15c-209fc78f969f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:39.625436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc967-d754-4a91-b15c-209fc78f969f None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:39.627022 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 524/1060] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:39.632713 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8a472-d03c-450e-bc4e-34f9277820f3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:39.633180 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8a472-d03c-450e-bc4e-34f9277820f3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:39.633566 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8a472-d03c-450e-bc4e-34f9277820f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:39.653240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4ed8a472-d03c-450e-bc4e-34f9277820f3 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:39.654472 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4ed8a472-d03c-450e-bc4e-34f9277820f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:39.734332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 537/1061] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:39.756710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc967-d754-4a91-b15c-209fc78f969f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:39.895455 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc967-d754-4a91-b15c-209fc78f969f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:39.922423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:39.932621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e2da8-e225-4008-ab45-3883aa4a3df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:39.933270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2da8-e225-4008-ab45-3883aa4a3df4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:39.933585 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2da8-e225-4008-ab45-3883aa4a3df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:39.933858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2da8-e225-4008-ab45-3883aa4a3df4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:39.945833 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e2da8-e225-4008-ab45-3883aa4a3df4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-909e2da8-e225-4008-ab45-3883aa4a3df4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:31:39.950733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:39.951662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:39.952555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:39.952555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:39.952555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:39.952555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:39.952555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:31:39.952555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:31:39.952555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:31:39.952555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:31:39.952555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:31:39.952555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:31:39.952555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 10:31:39.952555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:39.952555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 525/1062] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Aug 3 10:31:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:39.959735 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8a472-d03c-450e-bc4e-34f9277820f3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:40.063541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8a472-d03c-450e-bc4e-34f9277820f3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:40.087511 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:40.105044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bddfbd-03d7-4ff1-9293-d067bb897e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:40.105758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bddfbd-03d7-4ff1-9293-d067bb897e9f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:40.106037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bddfbd-03d7-4ff1-9293-d067bb897e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:40.106309 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bddfbd-03d7-4ff1-9293-d067bb897e9f None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:40.107039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bddfbd-03d7-4ff1-9293-d067bb897e9f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:40.107342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bddfbd-03d7-4ff1-9293-d067bb897e9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:40.108733 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bddfbd-03d7-4ff1-9293-d067bb897e9f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:40.123680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 538/1063] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Aug 3 10:31:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:40.125435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e2da8-e225-4008-ab45-3883aa4a3df4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:40.171786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df638e90-a510-49ba-bf8c-8ead2cdcbdb7 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:40.171964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df638e90-a510-49ba-bf8c-8ead2cdcbdb7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:40.172249 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df638e90-a510-49ba-bf8c-8ead2cdcbdb7 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:40.174439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 539/1064] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:40.179945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eeb620-5b9b-4254-b359-2ef888f417b0 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:40.180441 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eeb620-5b9b-4254-b359-2ef888f417b0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:40.180727 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eeb620-5b9b-4254-b359-2ef888f417b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:40.200129 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-93eeb620-5b9b-4254-b359-2ef888f417b0 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:40.201390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-93eeb620-5b9b-4254-b359-2ef888f417b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:40.243226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e2da8-e225-4008-ab45-3883aa4a3df4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:40.278360 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 540/1065] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:40.282359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd6f40-0a3a-40a5-88a2-7a2fc1181226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:40.283539 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd6f40-0a3a-40a5-88a2-7a2fc1181226 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:40.283819 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd6f40-0a3a-40a5-88a2-7a2fc1181226 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:40.284092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd6f40-0a3a-40a5-88a2-7a2fc1181226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:40.294603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd6f40-0a3a-40a5-88a2-7a2fc1181226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:40.296045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd6f40-0a3a-40a5-88a2-7a2fc1181226 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:40.302540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eeb620-5b9b-4254-b359-2ef888f417b0 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:40.312023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-8bbd6f40-0a3a-40a5-88a2-7a2fc1181226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:40.312023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:40.312023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:40.312023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:40.312023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:40.312023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:40.312023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:40.312023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:40.312023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:40.312023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:40.312023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:40.312023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:40.312023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:40.312023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:40.312023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:40.312990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:40.314119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:40.314119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:40.314119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:40.314119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:40.314119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 526/1066] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:40.319001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd6f40-0a3a-40a5-88a2-7a2fc1181226 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:40.362894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:40.412136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eeb620-5b9b-4254-b359-2ef888f417b0 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:40.428312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:40.433181 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd6f40-0a3a-40a5-88a2-7a2fc1181226 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:40.456018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63cac3-320e-459a-897d-85a7682147bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:40.457033 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63cac3-320e-459a-897d-85a7682147bb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:40.457539 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63cac3-320e-459a-897d-85a7682147bb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:40.457820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63cac3-320e-459a-897d-85a7682147bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:40.472794 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd8d33-09c5-4166-9d4c-a9f7e9f21ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:40.473495 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8d33-09c5-4166-9d4c-a9f7e9f21ced None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:40.473738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8d33-09c5-4166-9d4c-a9f7e9f21ced None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:40.474012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8d33-09c5-4166-9d4c-a9f7e9f21ced None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:40.474708 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd8d33-09c5-4166-9d4c-a9f7e9f21ced None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:40.475019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd8d33-09c5-4166-9d4c-a9f7e9f21ced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:40.476367 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd8d33-09c5-4166-9d4c-a9f7e9f21ced None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:40.490659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 527/1067] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:40.496468 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd8d33-09c5-4166-9d4c-a9f7e9f21ced None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:40.537013 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63cac3-320e-459a-897d-85a7682147bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:40.538519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63cac3-320e-459a-897d-85a7682147bb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:40.600097 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:40.604021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd8d33-09c5-4166-9d4c-a9f7e9f21ced None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:40.639177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61543bb-8b60-46cb-8206-290aeeefbb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:40.639808 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61543bb-8b60-46cb-8206-290aeeefbb5e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:40.640102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61543bb-8b60-46cb-8206-290aeeefbb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:40.640377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61543bb-8b60-46cb-8206-290aeeefbb5e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:40.656108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61543bb-8b60-46cb-8206-290aeeefbb5e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:40.656297 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 541/1068] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:31:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 10:31:40.656703 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61543bb-8b60-46cb-8206-290aeeefbb5e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:40.657161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63cac3-320e-459a-897d-85a7682147bb None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-e61543bb-8b60-46cb-8206-290aeeefbb5e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:31:40.668486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:40.669638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:40.671350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:40.671350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:40.671350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:31:40.671350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:31:40.671350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:31:40.671350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:31:40.671350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:31:40.671350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:31:40.671350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:31:40.671350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:40.671350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 528/1069] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 3 10:31:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:40.674404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:40.675866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61543bb-8b60-46cb-8206-290aeeefbb5e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:40.774772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63cac3-320e-459a-897d-85a7682147bb None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:40.787475 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61543bb-8b60-46cb-8206-290aeeefbb5e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:40.811565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc215d-3e0b-41d4-a008-0055d308f03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:40.812214 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc215d-3e0b-41d4-a008-0055d308f03e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:40.812494 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc215d-3e0b-41d4-a008-0055d308f03e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:40.812765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc215d-3e0b-41d4-a008-0055d308f03e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:40.823617 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc215d-3e0b-41d4-a008-0055d308f03e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:40.824010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde7056-0a91-433f-8618-1d769a7f5c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:40.824647 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde7056-0a91-433f-8618-1d769a7f5c13 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:40.824931 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde7056-0a91-433f-8618-1d769a7f5c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:40.825093 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc215d-3e0b-41d4-a008-0055d308f03e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:40.825253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde7056-0a91-433f-8618-1d769a7f5c13 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:40.826035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde7056-0a91-433f-8618-1d769a7f5c13 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:40.826346 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde7056-0a91-433f-8618-1d769a7f5c13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:40.827807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde7056-0a91-433f-8618-1d769a7f5c13 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:40.841336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-f2fc215d-3e0b-41d4-a008-0055d308f03e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:40.841336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:40.841336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:40.841336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:40.841336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:40.841336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:40.841336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:40.841336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:40.841336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:40.841336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:40.841336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:40.841336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:40.841336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:40.841336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:40.841336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:40.842437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:40.848632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:40.848632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:40.848632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:40.848632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:40.848632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 542/1070] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:40.850896 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc215d-3e0b-41d4-a008-0055d308f03e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:40.853039 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 529/1071] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 3 10:31:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:40.858570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde7056-0a91-433f-8618-1d769a7f5c13 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:40.878866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:40.916238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:40.959637 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc215d-3e0b-41d4-a008-0055d308f03e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:40.972202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde7056-0a91-433f-8618-1d769a7f5c13 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:40.997255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0804d2c-a4e1-483d-b65d-296c8c458d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:40.997932 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0804d2c-a4e1-483d-b65d-296c8c458d57 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:40.998305 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0804d2c-a4e1-483d-b65d-296c8c458d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:40.998483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0804d2c-a4e1-483d-b65d-296c8c458d57 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:40.999194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0804d2c-a4e1-483d-b65d-296c8c458d57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:40.999500 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0804d2c-a4e1-483d-b65d-296c8c458d57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:41.000862 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0804d2c-a4e1-483d-b65d-296c8c458d57 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:41.007803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4ea71-6ebc-4dd9-9491-5445d41e4c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:41.008930 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4ea71-6ebc-4dd9-9491-5445d41e4c9e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:41.009222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4ea71-6ebc-4dd9-9491-5445d41e4c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:41.009533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4ea71-6ebc-4dd9-9491-5445d41e4c9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:41.013502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 543/1072] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:41.025936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4ea71-6ebc-4dd9-9491-5445d41e4c9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:41.027528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4ea71-6ebc-4dd9-9491-5445d41e4c9e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:41.044663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-80e4ea71-6ebc-4dd9-9491-5445d41e4c9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:41.044663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:41.044663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:41.044663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:41.044663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:41.044663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:41.044663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:41.044663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:41.044663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:41.044663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:41.044663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:41.044663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:41.044663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:41.044663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:41.044663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:41.045965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:41.046887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:41.046887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:41.046887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:41.046887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:41.047527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 530/1073] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:41.051913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0804d2c-a4e1-483d-b65d-296c8c458d57 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:41.164465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0804d2c-a4e1-483d-b65d-296c8c458d57 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:41.191271 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:41.203831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b09ac0-e306-4664-8bbf-72288f557d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:41.205100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b09ac0-e306-4664-8bbf-72288f557d52 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:41.205382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b09ac0-e306-4664-8bbf-72288f557d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:41.205711 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b09ac0-e306-4664-8bbf-72288f557d52 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:41.206432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b09ac0-e306-4664-8bbf-72288f557d52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:41.206741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b09ac0-e306-4664-8bbf-72288f557d52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:41.208161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b09ac0-e306-4664-8bbf-72288f557d52 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:41.222239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 544/1074] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:41.228411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4ea71-6ebc-4dd9-9491-5445d41e4c9e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:41.301200 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:41.336019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4ea71-6ebc-4dd9-9491-5445d41e4c9e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:41.349171 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed201c3-96c5-492c-aeb3-4c5e25340486 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:41.349491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed201c3-96c5-492c-aeb3-4c5e25340486 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:41.349764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed201c3-96c5-492c-aeb3-4c5e25340486 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:41.351122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 545/1075] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:31:41.356034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526ced2-4a3d-4a94-aeea-cd14fbf702f8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:41.356328 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526ced2-4a3d-4a94-aeea-cd14fbf702f8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:41.357036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526ced2-4a3d-4a94-aeea-cd14fbf702f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:41.370533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754de8bb-015c-4708-97e0-1055ba28f534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:41.371177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754de8bb-015c-4708-97e0-1055ba28f534 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:41.371458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754de8bb-015c-4708-97e0-1055ba28f534 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:41.371730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754de8bb-015c-4708-97e0-1055ba28f534 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:41.375312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f526ced2-4a3d-4a94-aeea-cd14fbf702f8 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:41.377179 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f526ced2-4a3d-4a94-aeea-cd14fbf702f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:41.378842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754de8bb-015c-4708-97e0-1055ba28f534 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:41.380268 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754de8bb-015c-4708-97e0-1055ba28f534 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:41.396253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-754de8bb-015c-4708-97e0-1055ba28f534 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:41.396253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:41.396253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:41.396253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:41.396253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:41.396253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:41.396253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:41.396253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:41.396253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:41.396253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:41.396253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:41.396253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:41.396253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:31:41.396253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:31:41.396253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:41.397211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:41.398160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:31:41.398160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:31:41.398160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:31:41.398160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:31:41.398160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:31:41.398160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:31:41.398160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:41.398160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:41.398160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 531/1076] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:41.403235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754de8bb-015c-4708-97e0-1055ba28f534 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:41.450753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 546/1077] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:41.472958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526ced2-4a3d-4a94-aeea-cd14fbf702f8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:41.504294 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754de8bb-015c-4708-97e0-1055ba28f534 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:41.529234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:41.537354 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc1573-5d75-413f-a0b7-0ce33caa2dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:41.538007 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc1573-5d75-413f-a0b7-0ce33caa2dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:41.538290 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc1573-5d75-413f-a0b7-0ce33caa2dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:41.538566 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc1573-5d75-413f-a0b7-0ce33caa2dc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:41.539321 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc1573-5d75-413f-a0b7-0ce33caa2dc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:41.539624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc1573-5d75-413f-a0b7-0ce33caa2dc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:41.541051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc1573-5d75-413f-a0b7-0ce33caa2dc6 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:41.554774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 532/1078] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:31:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:41.560841 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc1573-5d75-413f-a0b7-0ce33caa2dc6 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:41.570603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526ced2-4a3d-4a94-aeea-cd14fbf702f8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:41.585063 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:41.606094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67166546-719c-4930-bcde-c193a689996e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:41.606736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67166546-719c-4930-bcde-c193a689996e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:41.607015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67166546-719c-4930-bcde-c193a689996e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:41.607295 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67166546-719c-4930-bcde-c193a689996e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:41.617752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67166546-719c-4930-bcde-c193a689996e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-67166546-719c-4930-bcde-c193a689996e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:31:41.622586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:41.623652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:41.624541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:41.624541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:41.624541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:41.624541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:41.624541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:31:41.624541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:31:41.624541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:31:41.624541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:31:41.624541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:31:41.624541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:31:41.624541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:31:41.624541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:41.624541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 547/1079] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:31:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:41.629981 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67166546-719c-4930-bcde-c193a689996e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:41.662391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc1573-5d75-413f-a0b7-0ce33caa2dc6 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:41.696955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978c834-6c03-4bd3-8552-dad12ced14d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:41.697672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978c834-6c03-4bd3-8552-dad12ced14d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:41.697954 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978c834-6c03-4bd3-8552-dad12ced14d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:41.698222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978c834-6c03-4bd3-8552-dad12ced14d8 None admin] PATH_INFO: `/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/e66f25cc329f418eba6bd637bf3e8a3a/roles/485bbc11b43a4e36b7ccb740582ec8d2` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:41.700425 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978c834-6c03-4bd3-8552-dad12ced14d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=4272b859979d49c8b8f0c17ec9eb81bc, user_id=e66f25cc329f418eba6bd637bf3e8a3a, role_id=485bbc11b43a4e36b7ccb740582ec8d2)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:41.701887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978c834-6c03-4bd3-8552-dad12ced14d8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:41.715336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 533/1080] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:31:41 2022] PUT /identity/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/e66f25cc329f418eba6bd637bf3e8a3a/roles/485bbc11b43a4e36b7ccb740582ec8d2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:31:41.736808 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67166546-719c-4930-bcde-c193a689996e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:41.780105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5a9d5-6849-4723-b41e-7bf235434035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:41.780744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5a9d5-6849-4723-b41e-7bf235434035 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:41.781028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5a9d5-6849-4723-b41e-7bf235434035 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:41.781304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5a9d5-6849-4723-b41e-7bf235434035 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:41.782078 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5a9d5-6849-4723-b41e-7bf235434035 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:41.782388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5a9d5-6849-4723-b41e-7bf235434035 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:41.784446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5a9d5-6849-4723-b41e-7bf235434035 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:41.796018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 548/1081] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:31:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:41.803096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978c834-6c03-4bd3-8552-dad12ced14d8 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:41.855841 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:41.910416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978c834-6c03-4bd3-8552-dad12ced14d8 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:41.944435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb326b2-0f60-41ec-a364-3529d693fe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:41.945083 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb326b2-0f60-41ec-a364-3529d693fe5d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:41.945359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb326b2-0f60-41ec-a364-3529d693fe5d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:41.945681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb326b2-0f60-41ec-a364-3529d693fe5d None admin] PATH_INFO: `/v3/users/ac8bf30fd2ea48d5b41389a2bd0a755d` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:41.946899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb326b2-0f60-41ec-a364-3529d693fe5d None admin] RBAC: Authorizing `identity:get_user(user_id=ac8bf30fd2ea48d5b41389a2bd0a755d)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:41.948351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb326b2-0f60-41ec-a364-3529d693fe5d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:41.955293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 534/1082] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:31:41 2022] GET /identity/v3/users/ac8bf30fd2ea48d5b41389a2bd0a755d => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:41.956734 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5a9d5-6849-4723-b41e-7bf235434035 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:41.993132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:42.056952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5a9d5-6849-4723-b41e-7bf235434035 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:42.092615 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03b92c-f570-422a-ac21-e0e25fab9954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:42.093273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03b92c-f570-422a-ac21-e0e25fab9954 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:42.093589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03b92c-f570-422a-ac21-e0e25fab9954 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:42.093862 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03b92c-f570-422a-ac21-e0e25fab9954 None admin] PATH_INFO: `/v3/projects/8a2b0c845b564700904b853a303f55e7` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:42.097203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03b92c-f570-422a-ac21-e0e25fab9954 None admin] RBAC: Authorizing `identity:get_project(project_id=8a2b0c845b564700904b853a303f55e7)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:42.098713 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03b92c-f570-422a-ac21-e0e25fab9954 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:42.100459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 549/1083] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:31:41 2022] GET /identity/v3/projects/8a2b0c845b564700904b853a303f55e7 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:42.106747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb326b2-0f60-41ec-a364-3529d693fe5d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:42.210173 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb326b2-0f60-41ec-a364-3529d693fe5d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:42.229599 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:42.248626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920bff3-0adb-450f-bf32-28dd9a11a890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:42.249224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920bff3-0adb-450f-bf32-28dd9a11a890 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:42.249539 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920bff3-0adb-450f-bf32-28dd9a11a890 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:42.249873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920bff3-0adb-450f-bf32-28dd9a11a890 None admin] PATH_INFO: `/v3/projects/8a2b0c845b564700904b853a303f55e7/users/ac8bf30fd2ea48d5b41389a2bd0a755d/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:42.251854 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920bff3-0adb-450f-bf32-28dd9a11a890 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a2b0c845b564700904b853a303f55e7, user_id=ac8bf30fd2ea48d5b41389a2bd0a755d, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:42.253298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920bff3-0adb-450f-bf32-28dd9a11a890 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:42.266452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 535/1084] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:31:42 2022] PUT /identity/v3/projects/8a2b0c845b564700904b853a303f55e7/users/ac8bf30fd2ea48d5b41389a2bd0a755d/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:31:43.696438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890ecff-de86-4739-ad7d-6cbbd5dc7217 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:43.696917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890ecff-de86-4739-ad7d-6cbbd5dc7217 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:43.697136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890ecff-de86-4739-ad7d-6cbbd5dc7217 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:43.698655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 550/1085] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:43.704107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c05152-235c-499a-a2af-35464d6c001d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:43.705050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c05152-235c-499a-a2af-35464d6c001d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:43.705307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c05152-235c-499a-a2af-35464d6c001d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:43.724400 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-27c05152-235c-499a-a2af-35464d6c001d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:43.725646 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-27c05152-235c-499a-a2af-35464d6c001d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:43.794010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 536/1086] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:43.817187 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890ecff-de86-4739-ad7d-6cbbd5dc7217 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:43.916253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890ecff-de86-4739-ad7d-6cbbd5dc7217 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:43.918547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:43.950687 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6de8b-91db-49f4-8c8f-930d4d316c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:43.951336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6de8b-91db-49f4-8c8f-930d4d316c6d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:43.951619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6de8b-91db-49f4-8c8f-930d4d316c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:43.951895 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6de8b-91db-49f4-8c8f-930d4d316c6d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:43.962551 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6de8b-91db-49f4-8c8f-930d4d316c6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-dcd6de8b-91db-49f4-8c8f-930d4d316c6d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:31:43.967796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:43.968801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:43.969791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:43.969791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:43.969791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:43.969791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:43.969791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:31:43.969791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:31:43.969791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:31:43.969791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:31:43.969791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:31:43.969791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:31:43.969791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 10:31:43.969791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:43.969791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 551/1087] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Aug 3 10:31:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:43.974957 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c05152-235c-499a-a2af-35464d6c001d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:44.072953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c05152-235c-499a-a2af-35464d6c001d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:44.080832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:44.108621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4b1f8-85ff-473c-a8e4-e5c221408b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:44.109252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4b1f8-85ff-473c-a8e4-e5c221408b9e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:44.109564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4b1f8-85ff-473c-a8e4-e5c221408b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:44.109843 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4b1f8-85ff-473c-a8e4-e5c221408b9e None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:44.110605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4b1f8-85ff-473c-a8e4-e5c221408b9e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:44.110916 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4b1f8-85ff-473c-a8e4-e5c221408b9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:44.112299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4b1f8-85ff-473c-a8e4-e5c221408b9e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:44.120257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 537/1088] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Aug 3 10:31:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:44.125922 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6de8b-91db-49f4-8c8f-930d4d316c6d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:44.226123 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6de8b-91db-49f4-8c8f-930d4d316c6d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:44.260835 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd1f0c-9f92-4219-af89-bbe020dd1000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:44.261488 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd1f0c-9f92-4219-af89-bbe020dd1000 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:44.261801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd1f0c-9f92-4219-af89-bbe020dd1000 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:44.262072 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd1f0c-9f92-4219-af89-bbe020dd1000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:44.272260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd1f0c-9f92-4219-af89-bbe020dd1000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:44.273267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e6de9-1b6d-4293-9dbe-02a35ee044d8 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:44.273591 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e6de9-1b6d-4293-9dbe-02a35ee044d8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:44.273722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd1f0c-9f92-4219-af89-bbe020dd1000 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:44.273878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e6de9-1b6d-4293-9dbe-02a35ee044d8 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:44.275236 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 538/1089] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:31:44.280288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c0fa6-7e39-48fa-a12f-23be6ea24134 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:44.280575 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c0fa6-7e39-48fa-a12f-23be6ea24134 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:44.280848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c0fa6-7e39-48fa-a12f-23be6ea24134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:44.289660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-6ddd1f0c-9f92-4219-af89-bbe020dd1000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:44.289660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:44.289660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:44.289660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:44.289660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:44.289660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:44.289660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:44.289660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:44.289660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:44.289660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:44.289660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:44.289660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:44.289660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:44.289660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:44.289660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:44.290760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:44.292099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:44.292099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:44.292099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:44.292099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:44.292099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 552/1090] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:44.296443 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd1f0c-9f92-4219-af89-bbe020dd1000 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:44.299411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f62c0fa6-7e39-48fa-a12f-23be6ea24134 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:44.300606 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f62c0fa6-7e39-48fa-a12f-23be6ea24134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:44.315870 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:44.368519 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 539/1091] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:44.391488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c0fa6-7e39-48fa-a12f-23be6ea24134 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:44.398701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd1f0c-9f92-4219-af89-bbe020dd1000 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:44.432353 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fde46-0ec0-487b-b5d0-474a25e048db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:44.432980 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fde46-0ec0-487b-b5d0-474a25e048db None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:44.433266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fde46-0ec0-487b-b5d0-474a25e048db None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:44.433575 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fde46-0ec0-487b-b5d0-474a25e048db None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:44.434280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fde46-0ec0-487b-b5d0-474a25e048db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:44.434590 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fde46-0ec0-487b-b5d0-474a25e048db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:44.435957 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fde46-0ec0-487b-b5d0-474a25e048db None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:44.448162 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 553/1092] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:44.453778 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fde46-0ec0-487b-b5d0-474a25e048db None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:44.487217 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:44.491723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c0fa6-7e39-48fa-a12f-23be6ea24134 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:44.525793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb0bea-e925-4546-8ada-1eba5faac2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:44.526446 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb0bea-e925-4546-8ada-1eba5faac2dc None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:44.526729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb0bea-e925-4546-8ada-1eba5faac2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:44.527009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb0bea-e925-4546-8ada-1eba5faac2dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:44.529902 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:44.538794 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb0bea-e925-4546-8ada-1eba5faac2dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-29eb0bea-e925-4546-8ada-1eba5faac2dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:31:44.543724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:44.544723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:44.545607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:44.545607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:44.545607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:44.545607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:44.545607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:31:44.545607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:31:44.545607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:31:44.545607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:31:44.545607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:31:44.545607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:31:44.545607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:31:44.545607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:31:44.545607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 540/1093] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:31:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:44.550835 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb0bea-e925-4546-8ada-1eba5faac2dc None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:44.558160 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fde46-0ec0-487b-b5d0-474a25e048db None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:44.593087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839b92e-1953-401f-8398-3b9a825a2651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:44.593769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839b92e-1953-401f-8398-3b9a825a2651 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:44.594051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839b92e-1953-401f-8398-3b9a825a2651 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:44.594329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839b92e-1953-401f-8398-3b9a825a2651 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:44.601505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839b92e-1953-401f-8398-3b9a825a2651 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:44.602947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839b92e-1953-401f-8398-3b9a825a2651 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-6839b92e-1953-401f-8398-3b9a825a2651 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:31:44.614582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:44.615568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:44.616517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:44.616517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:44.616517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:31:44.616517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:31:44.616517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:31:44.616517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:31:44.616517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:31:44.616517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:31:44.616517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 10:31:44.616517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:44.616517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 554/1094] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 3 10:31:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:44.621668 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839b92e-1953-401f-8398-3b9a825a2651 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:44.656507 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb0bea-e925-4546-8ada-1eba5faac2dc None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:44.692632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47945836-7b9a-4d97-b26d-8b6ecbe257b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:44.693267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47945836-7b9a-4d97-b26d-8b6ecbe257b2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:44.693563 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47945836-7b9a-4d97-b26d-8b6ecbe257b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:44.693836 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47945836-7b9a-4d97-b26d-8b6ecbe257b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:44.694557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47945836-7b9a-4d97-b26d-8b6ecbe257b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:44.694855 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47945836-7b9a-4d97-b26d-8b6ecbe257b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:44.696218 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47945836-7b9a-4d97-b26d-8b6ecbe257b2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:44.705454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 541/1095] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:31:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:44.711256 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47945836-7b9a-4d97-b26d-8b6ecbe257b2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:44.731585 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839b92e-1953-401f-8398-3b9a825a2651 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:44.735675 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:44.771411 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538b2fb-5609-4a87-976a-744222468b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:44.771973 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538b2fb-5609-4a87-976a-744222468b44 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:44.772256 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538b2fb-5609-4a87-976a-744222468b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:44.772618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538b2fb-5609-4a87-976a-744222468b44 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:44.773786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538b2fb-5609-4a87-976a-744222468b44 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:44.774235 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538b2fb-5609-4a87-976a-744222468b44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:44.775703 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538b2fb-5609-4a87-976a-744222468b44 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:44.778720 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:44.795305 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 555/1096] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 3 10:31:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:44.800998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538b2fb-5609-4a87-976a-744222468b44 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:44.828900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47945836-7b9a-4d97-b26d-8b6ecbe257b2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:44.866013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae4010-89ac-494b-831f-403991105473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:44.866661 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae4010-89ac-494b-831f-403991105473 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:44.866941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae4010-89ac-494b-831f-403991105473 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:44.867223 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae4010-89ac-494b-831f-403991105473 None admin] PATH_INFO: `/v3/users/ac8bf30fd2ea48d5b41389a2bd0a755d` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:44.868397 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae4010-89ac-494b-831f-403991105473 None admin] RBAC: Authorizing `identity:get_user(user_id=ac8bf30fd2ea48d5b41389a2bd0a755d)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:44.869849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae4010-89ac-494b-831f-403991105473 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:44.871473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 542/1097] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:31:44 2022] GET /identity/v3/users/ac8bf30fd2ea48d5b41389a2bd0a755d => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:44.877857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae4010-89ac-494b-831f-403991105473 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:44.906325 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538b2fb-5609-4a87-976a-744222468b44 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:44.942287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315ee95-b278-4834-8ff2-5c3bb40fee26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:44.942936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315ee95-b278-4834-8ff2-5c3bb40fee26 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:44.943218 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315ee95-b278-4834-8ff2-5c3bb40fee26 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:44.943491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315ee95-b278-4834-8ff2-5c3bb40fee26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:44.967754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:44.982056 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae4010-89ac-494b-831f-403991105473 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:45.007512 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:45.015335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315ee95-b278-4834-8ff2-5c3bb40fee26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:45.016796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315ee95-b278-4834-8ff2-5c3bb40fee26 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:45.018510 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2facd0-48ef-4221-8c7f-b3829d0ae071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:45.019141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2facd0-48ef-4221-8c7f-b3829d0ae071 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:45.019429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2facd0-48ef-4221-8c7f-b3829d0ae071 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:45.019707 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2facd0-48ef-4221-8c7f-b3829d0ae071 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:45.091088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2facd0-48ef-4221-8c7f-b3829d0ae071 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:45.092416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2facd0-48ef-4221-8c7f-b3829d0ae071 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:45.120209 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 556/1098] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:31:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 10:31:45.126106 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315ee95-b278-4834-8ff2-5c3bb40fee26 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:45.188014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:45.202860 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 543/1099] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:31:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 10:31:45.208912 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2facd0-48ef-4221-8c7f-b3829d0ae071 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:45.231558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:45.240224 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315ee95-b278-4834-8ff2-5c3bb40fee26 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:45.277064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd72b1-924d-4549-a17d-2d1b99095dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:45.277816 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd72b1-924d-4549-a17d-2d1b99095dce None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:45.278099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd72b1-924d-4549-a17d-2d1b99095dce None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:45.278374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd72b1-924d-4549-a17d-2d1b99095dce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:45.288699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd72b1-924d-4549-a17d-2d1b99095dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:45.290196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd72b1-924d-4549-a17d-2d1b99095dce None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:45.305584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-70fd72b1-924d-4549-a17d-2d1b99095dce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:45.305584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:45.305584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:45.305584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:45.305584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:45.305584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:45.305584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:45.305584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:45.305584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:45.305584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:45.305584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:45.305584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:45.305584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:31:45.305584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:31:45.305584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:45.306684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:45.307583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:31:45.307583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:31:45.307583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:31:45.307583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:45.307583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 557/1100] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:31:45.312673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd72b1-924d-4549-a17d-2d1b99095dce None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:45.325867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2facd0-48ef-4221-8c7f-b3829d0ae071 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:45.361453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314fa80-1df7-46c4-8c43-bd04b0b70e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:45.362054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314fa80-1df7-46c4-8c43-bd04b0b70e7d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:45.362334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314fa80-1df7-46c4-8c43-bd04b0b70e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:45.362610 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314fa80-1df7-46c4-8c43-bd04b0b70e7d None admin] PATH_INFO: `/v3/projects/8a2b0c845b564700904b853a303f55e7` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:45.363779 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314fa80-1df7-46c4-8c43-bd04b0b70e7d None admin] RBAC: Authorizing `identity:get_project(project_id=8a2b0c845b564700904b853a303f55e7)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:45.365207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314fa80-1df7-46c4-8c43-bd04b0b70e7d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:45.366914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 544/1101] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:31:45 2022] GET /identity/v3/projects/8a2b0c845b564700904b853a303f55e7 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:45.372920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314fa80-1df7-46c4-8c43-bd04b0b70e7d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:45.413260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd72b1-924d-4549-a17d-2d1b99095dce None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:45.435095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:45.465263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5241dd-aee1-4a75-9093-006642610011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:45.465888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5241dd-aee1-4a75-9093-006642610011 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:45.466158 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5241dd-aee1-4a75-9093-006642610011 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:45.466427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5241dd-aee1-4a75-9093-006642610011 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:45.467128 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5241dd-aee1-4a75-9093-006642610011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:45.467431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5241dd-aee1-4a75-9093-006642610011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:45.468972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5241dd-aee1-4a75-9093-006642610011 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:45.481216 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314fa80-1df7-46c4-8c43-bd04b0b70e7d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:45.481822 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 558/1102] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:45.487639 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5241dd-aee1-4a75-9093-006642610011 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:45.496581 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:45.518004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ae2b3-e13e-4064-b77f-5c97fd1277fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:45.518626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ae2b3-e13e-4064-b77f-5c97fd1277fc None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:45.518905 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ae2b3-e13e-4064-b77f-5c97fd1277fc None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:45.519178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ae2b3-e13e-4064-b77f-5c97fd1277fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:45.519956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ae2b3-e13e-4064-b77f-5c97fd1277fc None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=8a2b0c845b564700904b853a303f55e7, user.id=ac8bf30fd2ea48d5b41389a2bd0a755d) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:45.520260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ae2b3-e13e-4064-b77f-5c97fd1277fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:45.521702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ae2b3-e13e-4064-b77f-5c97fd1277fc None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:45.533573 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 545/1103] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:31:45 2022] GET /identity/v3/role_assignments?user.id=ac8bf30fd2ea48d5b41389a2bd0a755d&scope.project.id=8a2b0c845b564700904b853a303f55e7&role.id=01714797cab143e1a7a150c068fa9315 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:45.623328 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5241dd-aee1-4a75-9093-006642610011 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:45.676356 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6f049-dc5a-4d3d-8726-6ea70c5356f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:45.677011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6f049-dc5a-4d3d-8726-6ea70c5356f5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:45.677287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6f049-dc5a-4d3d-8726-6ea70c5356f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:45.677586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6f049-dc5a-4d3d-8726-6ea70c5356f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:45.685583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6f049-dc5a-4d3d-8726-6ea70c5356f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:45.687043 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6f049-dc5a-4d3d-8726-6ea70c5356f5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:45.699282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ecb6f049-dc5a-4d3d-8726-6ea70c5356f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:45.699282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:31:45.699282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:31:45.699282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:31:45.699282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:31:45.699282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:31:45.699282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:31:45.699282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:31:45.699282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:31:45.699282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:31:45.699282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:31:45.699282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:31:45.699282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:31:45.699282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:31:45.699282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:31:45.700257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:31:45.701163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:31:45.701163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:31:45.701163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:31:45.701163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:31:45.701163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:31:45.701163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:31:45.701163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:31:45.701163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:31:45.701163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 559/1104] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:31:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:31:45.706903 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ae2b3-e13e-4064-b77f-5c97fd1277fc None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:45.810605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ae2b3-e13e-4064-b77f-5c97fd1277fc None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:45.831489 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:45.846464 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b974d-ac80-4369-a457-8177025e8204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:45.847080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b974d-ac80-4369-a457-8177025e8204 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:45.847357 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b974d-ac80-4369-a457-8177025e8204 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:45.847630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b974d-ac80-4369-a457-8177025e8204 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:45.848380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b974d-ac80-4369-a457-8177025e8204 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:45.848679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b974d-ac80-4369-a457-8177025e8204 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:45.850130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b974d-ac80-4369-a457-8177025e8204 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:45.862195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 546/1105] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:31:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:45.868327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6f049-dc5a-4d3d-8726-6ea70c5356f5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:45.935092 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:45.973036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6f049-dc5a-4d3d-8726-6ea70c5356f5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:46.009307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f581b6-36dc-44f1-b2f9-1515703265f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:46.009981 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f581b6-36dc-44f1-b2f9-1515703265f6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:46.010258 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f581b6-36dc-44f1-b2f9-1515703265f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:46.010529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f581b6-36dc-44f1-b2f9-1515703265f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:46.011333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f581b6-36dc-44f1-b2f9-1515703265f6 None admin] RBAC: Adding query filter params (role.id=485bbc11b43a4e36b7ccb740582ec8d2, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=e66f25cc329f418eba6bd637bf3e8a3a) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:46.011640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f581b6-36dc-44f1-b2f9-1515703265f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:46.013088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f581b6-36dc-44f1-b2f9-1515703265f6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:46.020769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 560/1106] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:31:45 2022] GET /identity/v3/role_assignments?user.id=e66f25cc329f418eba6bd637bf3e8a3a&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=485bbc11b43a4e36b7ccb740582ec8d2 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:47.581695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922d903-549a-4f12-a430-82d96ee4ccd5 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:47.582218 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922d903-549a-4f12-a430-82d96ee4ccd5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:47.582338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922d903-549a-4f12-a430-82d96ee4ccd5 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:47.583596 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 547/1107] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:47.588708 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9f9d9-00c2-4c51-be85-273886993a27 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:47.589118 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9f9d9-00c2-4c51-be85-273886993a27 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:47.589250 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9f9d9-00c2-4c51-be85-273886993a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:47.608440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c7a9f9d9-00c2-4c51-be85-273886993a27 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:47.609672 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c7a9f9d9-00c2-4c51-be85-273886993a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:47.675125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 561/1108] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:47.699847 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922d903-549a-4f12-a430-82d96ee4ccd5 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:47.809706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922d903-549a-4f12-a430-82d96ee4ccd5 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:47.844419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc92c56-4892-4fc1-9ccf-bb484366bfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:47.845010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc92c56-4892-4fc1-9ccf-bb484366bfc3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:47.845292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc92c56-4892-4fc1-9ccf-bb484366bfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:47.845596 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc92c56-4892-4fc1-9ccf-bb484366bfc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:47.846738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc92c56-4892-4fc1-9ccf-bb484366bfc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:47.848150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc92c56-4892-4fc1-9ccf-bb484366bfc3 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:47.849770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 548/1109] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:47.855648 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9f9d9-00c2-4c51-be85-273886993a27 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:47.911386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:47.950368 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9f9d9-00c2-4c51-be85-273886993a27 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:47.983793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f652384-9521-443b-8e8a-ecdd06c5c49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:47.984413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f652384-9521-443b-8e8a-ecdd06c5c49e None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:47.984690 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f652384-9521-443b-8e8a-ecdd06c5c49e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:47.984956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f652384-9521-443b-8e8a-ecdd06c5c49e None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:47.986225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f652384-9521-443b-8e8a-ecdd06c5c49e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:47.987645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f652384-9521-443b-8e8a-ecdd06c5c49e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:48.008457 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 562/1110] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:31:47 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:31:50.034082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f04d3f-d8e6-4f05-be17-00ef978a22a5 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:50.034659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f04d3f-d8e6-4f05-be17-00ef978a22a5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:50.034659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f04d3f-d8e6-4f05-be17-00ef978a22a5 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:50.035981 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 549/1111] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:50.041032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60bb25-c699-480f-92f4-5097896a9b59 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:50.041406 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60bb25-c699-480f-92f4-5097896a9b59 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:50.041607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60bb25-c699-480f-92f4-5097896a9b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:50.061756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2b60bb25-c699-480f-92f4-5097896a9b59 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:50.062964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2b60bb25-c699-480f-92f4-5097896a9b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:50.126996 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 563/1112] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:31:50.149729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f04d3f-d8e6-4f05-be17-00ef978a22a5 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:50.198604 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:50.249374 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f04d3f-d8e6-4f05-be17-00ef978a22a5 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:50.282232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ea286-b97e-4372-be59-4db94b56e8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:50.283423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ea286-b97e-4372-be59-4db94b56e8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:50.283702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ea286-b97e-4372-be59-4db94b56e8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:50.283972 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ea286-b97e-4372-be59-4db94b56e8c9 None admin] PATH_INFO: `/v3/roles/cde4978cbb9648639ff720b5ca82886b` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:50.285453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ea286-b97e-4372-be59-4db94b56e8c9 None admin] RBAC: Authorizing `identity:get_role(role_id=cde4978cbb9648639ff720b5ca82886b)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:50.286793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ea286-b97e-4372-be59-4db94b56e8c9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:50.288149 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 550/1113] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:31:50 2022] GET /identity/v3/roles/cde4978cbb9648639ff720b5ca82886b => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:50.294088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60bb25-c699-480f-92f4-5097896a9b59 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:50.340044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:50.391258 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60bb25-c699-480f-92f4-5097896a9b59 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:50.425262 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed2320-8d58-4a6f-8e3b-2a574a8138dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:50.425941 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed2320-8d58-4a6f-8e3b-2a574a8138dd None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:50.426222 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed2320-8d58-4a6f-8e3b-2a574a8138dd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:50.426501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed2320-8d58-4a6f-8e3b-2a574a8138dd None admin] PATH_INFO: `/v3/users/a0981f086f5544bcbde44cf8689bb0b0` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:50.439616 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed2320-8d58-4a6f-8e3b-2a574a8138dd None admin] RBAC: Authorizing `identity:get_user(user_id=a0981f086f5544bcbde44cf8689bb0b0)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:50.441087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed2320-8d58-4a6f-8e3b-2a574a8138dd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:50.442776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 564/1114] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:31:50 2022] GET /identity/v3/users/a0981f086f5544bcbde44cf8689bb0b0 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:50.448513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ea286-b97e-4372-be59-4db94b56e8c9 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:50.554461 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ea286-b97e-4372-be59-4db94b56e8c9 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:50.570937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:50.590082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf7c0a-7260-4a98-8da0-6bf726e77a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:50.590709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf7c0a-7260-4a98-8da0-6bf726e77a8f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:50.590996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf7c0a-7260-4a98-8da0-6bf726e77a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:50.591270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf7c0a-7260-4a98-8da0-6bf726e77a8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:50.656681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf7c0a-7260-4a98-8da0-6bf726e77a8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:50.658191 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf7c0a-7260-4a98-8da0-6bf726e77a8f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:50.755127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 551/1115] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:31:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 10:31:50.761053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed2320-8d58-4a6f-8e3b-2a574a8138dd None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:50.863007 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed2320-8d58-4a6f-8e3b-2a574a8138dd None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:50.865704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:50.898311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74d589-e662-47ef-afd4-36add339c6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:50.898986 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74d589-e662-47ef-afd4-36add339c6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:50.899267 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74d589-e662-47ef-afd4-36add339c6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:50.899539 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74d589-e662-47ef-afd4-36add339c6e7 None admin] PATH_INFO: `/v3/projects/8a2b0c845b564700904b853a303f55e7` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:50.900702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74d589-e662-47ef-afd4-36add339c6e7 None admin] RBAC: Authorizing `identity:get_project(project_id=8a2b0c845b564700904b853a303f55e7)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:50.902155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74d589-e662-47ef-afd4-36add339c6e7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:50.903830 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 565/1116] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:31:50 2022] GET /identity/v3/projects/8a2b0c845b564700904b853a303f55e7 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:50.910509 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf7c0a-7260-4a98-8da0-6bf726e77a8f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:50.927749 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:51.008936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf7c0a-7260-4a98-8da0-6bf726e77a8f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:51.041144 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5613f-b358-4399-82fc-d8055415c5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:51.041836 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5613f-b358-4399-82fc-d8055415c5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:51.042050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5613f-b358-4399-82fc-d8055415c5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:51.042322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5613f-b358-4399-82fc-d8055415c5c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:51.043099 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5613f-b358-4399-82fc-d8055415c5c9 None admin] RBAC: Adding query filter params (role.id=cde4978cbb9648639ff720b5ca82886b, scope.project.id=8a2b0c845b564700904b853a303f55e7, user.id=a0981f086f5544bcbde44cf8689bb0b0) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:51.043406 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5613f-b358-4399-82fc-d8055415c5c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:51.044948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5613f-b358-4399-82fc-d8055415c5c9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:51.050917 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 552/1117] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:31:50 2022] GET /identity/v3/role_assignments?user.id=a0981f086f5544bcbde44cf8689bb0b0&scope.project.id=8a2b0c845b564700904b853a303f55e7&role.id=cde4978cbb9648639ff720b5ca82886b => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:53.043523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e27f2d-683e-4298-b564-19620ad0cbc0 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:53.044085 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e27f2d-683e-4298-b564-19620ad0cbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:53.044085 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e27f2d-683e-4298-b564-19620ad0cbc0 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:53.045393 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 566/1118] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:53.051122 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f798eb-743d-4f9f-b111-baeed19bdf6d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:53.051565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f798eb-743d-4f9f-b111-baeed19bdf6d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:53.051659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f798eb-743d-4f9f-b111-baeed19bdf6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:53.069577 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c3f798eb-743d-4f9f-b111-baeed19bdf6d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:53.070778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c3f798eb-743d-4f9f-b111-baeed19bdf6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:53.145274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 553/1119] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:53.168403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e27f2d-683e-4298-b564-19620ad0cbc0 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:53.269008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e27f2d-683e-4298-b564-19620ad0cbc0 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:53.303320 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0721aa-779b-4c85-aaa6-00ff2ca68de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:53.303882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0721aa-779b-4c85-aaa6-00ff2ca68de8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:53.304160 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0721aa-779b-4c85-aaa6-00ff2ca68de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:53.304432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0721aa-779b-4c85-aaa6-00ff2ca68de8 None admin] PATH_INFO: `/v3/roles/cde4978cbb9648639ff720b5ca82886b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:53.305887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0721aa-779b-4c85-aaa6-00ff2ca68de8 None admin] RBAC: Authorizing `identity:get_role(role_id=cde4978cbb9648639ff720b5ca82886b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:53.307239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0721aa-779b-4c85-aaa6-00ff2ca68de8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:53.308592 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 567/1120] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:31:53 2022] GET /identity/v3/roles/cde4978cbb9648639ff720b5ca82886b => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:53.314589 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f798eb-743d-4f9f-b111-baeed19bdf6d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:53.326869 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:53.413159 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f798eb-743d-4f9f-b111-baeed19bdf6d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:53.446296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be72e6-8253-4c9c-8427-76a889bad580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:53.446942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be72e6-8253-4c9c-8427-76a889bad580 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:53.447217 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be72e6-8253-4c9c-8427-76a889bad580 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:53.447487 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be72e6-8253-4c9c-8427-76a889bad580 None admin] PATH_INFO: `/v3/users/a0981f086f5544bcbde44cf8689bb0b0` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:53.448623 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be72e6-8253-4c9c-8427-76a889bad580 None admin] RBAC: Authorizing `identity:get_user(user_id=a0981f086f5544bcbde44cf8689bb0b0)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:53.450074 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be72e6-8253-4c9c-8427-76a889bad580 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:53.451682 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 554/1121] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:31:53 2022] GET /identity/v3/users/a0981f086f5544bcbde44cf8689bb0b0 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:53.457515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0721aa-779b-4c85-aaa6-00ff2ca68de8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:53.507434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:53.565748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0721aa-779b-4c85-aaa6-00ff2ca68de8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:53.599212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbc268-24d7-468a-ae8a-37784b16953d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:53.599856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbc268-24d7-468a-ae8a-37784b16953d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:53.600136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbc268-24d7-468a-ae8a-37784b16953d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:53.600408 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbc268-24d7-468a-ae8a-37784b16953d None admin] PATH_INFO: `/v3/projects/8a2b0c845b564700904b853a303f55e7` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:53.601589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbc268-24d7-468a-ae8a-37784b16953d None admin] RBAC: Authorizing `identity:get_project(project_id=8a2b0c845b564700904b853a303f55e7)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:53.603011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbc268-24d7-468a-ae8a-37784b16953d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:53.604691 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 568/1122] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:31:53 2022] GET /identity/v3/projects/8a2b0c845b564700904b853a303f55e7 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:53.611378 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be72e6-8253-4c9c-8427-76a889bad580 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:53.687937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:53.709235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be72e6-8253-4c9c-8427-76a889bad580 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:53.743161 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638d014-ce05-4c16-8ab0-64050db04d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:53.743837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638d014-ce05-4c16-8ab0-64050db04d3e None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:53.744134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638d014-ce05-4c16-8ab0-64050db04d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:53.744397 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638d014-ce05-4c16-8ab0-64050db04d3e None admin] PATH_INFO: `/v3/projects/8a2b0c845b564700904b853a303f55e7/users/a0981f086f5544bcbde44cf8689bb0b0/roles/cde4978cbb9648639ff720b5ca82886b` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:53.746603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638d014-ce05-4c16-8ab0-64050db04d3e None admin] RBAC: Authorizing `identity:create_grant(project_id=8a2b0c845b564700904b853a303f55e7, user_id=a0981f086f5544bcbde44cf8689bb0b0, role_id=cde4978cbb9648639ff720b5ca82886b)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:53.748034 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638d014-ce05-4c16-8ab0-64050db04d3e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:53.761196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 555/1123] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:31:53 2022] PUT /identity/v3/projects/8a2b0c845b564700904b853a303f55e7/users/a0981f086f5544bcbde44cf8689bb0b0/roles/cde4978cbb9648639ff720b5ca82886b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:31:55.773961 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50c884-3468-4d95-bbda-08698b6456bb None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:55.774540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50c884-3468-4d95-bbda-08698b6456bb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:55.774540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50c884-3468-4d95-bbda-08698b6456bb None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:55.776466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 569/1124] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:55.782626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5265a99-cd04-4a14-909d-53b166f9d6fe None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:55.783105 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5265a99-cd04-4a14-909d-53b166f9d6fe None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:55.783208 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5265a99-cd04-4a14-909d-53b166f9d6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:55.803829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f5265a99-cd04-4a14-909d-53b166f9d6fe None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:55.805134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f5265a99-cd04-4a14-909d-53b166f9d6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:55.885558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 556/1125] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:55.913798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50c884-3468-4d95-bbda-08698b6456bb None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:56.038895 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50c884-3468-4d95-bbda-08698b6456bb None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:56.050953 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:56.079064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971587d5-2d9e-40ae-90bb-926b3321a1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:56.080394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971587d5-2d9e-40ae-90bb-926b3321a1ac None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:56.080678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971587d5-2d9e-40ae-90bb-926b3321a1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:56.081076 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971587d5-2d9e-40ae-90bb-926b3321a1ac None admin] PATH_INFO: `/v3/roles/cde4978cbb9648639ff720b5ca82886b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:56.083130 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971587d5-2d9e-40ae-90bb-926b3321a1ac None admin] RBAC: Authorizing `identity:get_role(role_id=cde4978cbb9648639ff720b5ca82886b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:56.084990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971587d5-2d9e-40ae-90bb-926b3321a1ac None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:56.087137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 570/1126] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:31:55 2022] GET /identity/v3/roles/cde4978cbb9648639ff720b5ca82886b => generated 223 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:56.094222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5265a99-cd04-4a14-909d-53b166f9d6fe None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:56.171090 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:56.208856 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5265a99-cd04-4a14-909d-53b166f9d6fe None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:56.247253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ca818-e24c-4cdc-9d22-a3b04bf75f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:56.247905 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ca818-e24c-4cdc-9d22-a3b04bf75f2e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:56.248178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ca818-e24c-4cdc-9d22-a3b04bf75f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:56.248498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ca818-e24c-4cdc-9d22-a3b04bf75f2e None admin] PATH_INFO: `/v3/users/a0981f086f5544bcbde44cf8689bb0b0` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:56.249859 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ca818-e24c-4cdc-9d22-a3b04bf75f2e None admin] RBAC: Authorizing `identity:get_user(user_id=a0981f086f5544bcbde44cf8689bb0b0)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:56.252111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ca818-e24c-4cdc-9d22-a3b04bf75f2e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:56.254120 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 557/1127] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:31:56 2022] GET /identity/v3/users/a0981f086f5544bcbde44cf8689bb0b0 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:56.260057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971587d5-2d9e-40ae-90bb-926b3321a1ac None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:56.372340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971587d5-2d9e-40ae-90bb-926b3321a1ac None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:56.406286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e11ed-0406-4692-a70c-735c6c2780d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:56.406908 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e11ed-0406-4692-a70c-735c6c2780d4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:56.407192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e11ed-0406-4692-a70c-735c6c2780d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:56.407499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e11ed-0406-4692-a70c-735c6c2780d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:56.455844 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:56.481271 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e11ed-0406-4692-a70c-735c6c2780d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:56.483006 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e11ed-0406-4692-a70c-735c6c2780d4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:56.590283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 571/1128] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:31:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 10:31:56.598045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ca818-e24c-4cdc-9d22-a3b04bf75f2e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:56.706149 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ca818-e24c-4cdc-9d22-a3b04bf75f2e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:56.742402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2cbb0-e1a1-4a4c-97e5-8b6bc4949e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:56.743061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2cbb0-e1a1-4a4c-97e5-8b6bc4949e35 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:56.743346 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2cbb0-e1a1-4a4c-97e5-8b6bc4949e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:56.743625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2cbb0-e1a1-4a4c-97e5-8b6bc4949e35 None admin] PATH_INFO: `/v3/projects/8a2b0c845b564700904b853a303f55e7` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:56.744927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2cbb0-e1a1-4a4c-97e5-8b6bc4949e35 None admin] RBAC: Authorizing `identity:get_project(project_id=8a2b0c845b564700904b853a303f55e7)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:56.746417 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2cbb0-e1a1-4a4c-97e5-8b6bc4949e35 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:56.748279 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 558/1129] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:31:56 2022] GET /identity/v3/projects/8a2b0c845b564700904b853a303f55e7 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:56.754579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e11ed-0406-4692-a70c-735c6c2780d4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:56.834418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:56.867319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e11ed-0406-4692-a70c-735c6c2780d4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:56.902762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09004ab-87d0-46e9-ab3b-c96f4ed75a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:56.903413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09004ab-87d0-46e9-ab3b-c96f4ed75a16 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:56.903676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09004ab-87d0-46e9-ab3b-c96f4ed75a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:56.903955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09004ab-87d0-46e9-ab3b-c96f4ed75a16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:56.904753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09004ab-87d0-46e9-ab3b-c96f4ed75a16 None admin] RBAC: Adding query filter params (role.id=cde4978cbb9648639ff720b5ca82886b, scope.project.id=8a2b0c845b564700904b853a303f55e7, user.id=a0981f086f5544bcbde44cf8689bb0b0) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:31:56.905064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09004ab-87d0-46e9-ab3b-c96f4ed75a16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:56.906658 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09004ab-87d0-46e9-ab3b-c96f4ed75a16 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:56.914475 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 572/1130] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:31:56 2022] GET /identity/v3/role_assignments?user.id=a0981f086f5544bcbde44cf8689bb0b0&scope.project.id=8a2b0c845b564700904b853a303f55e7&role.id=cde4978cbb9648639ff720b5ca82886b => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:31:59.005257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2793f4c-bc84-4451-a0dc-a025556e36c2 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:59.005851 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2793f4c-bc84-4451-a0dc-a025556e36c2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:59.005851 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2793f4c-bc84-4451-a0dc-a025556e36c2 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:59.007220 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 559/1131] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:31:59.012444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6436266-760f-4bf5-a935-567246b7b756 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:59.012867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6436266-760f-4bf5-a935-567246b7b756 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:59.012991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6436266-760f-4bf5-a935-567246b7b756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:59.031449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d6436266-760f-4bf5-a935-567246b7b756 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:31:59.033147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d6436266-760f-4bf5-a935-567246b7b756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:31:59.097783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 573/1132] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:31:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:31:59.121070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2793f4c-bc84-4451-a0dc-a025556e36c2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:59.124854 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:59.236128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2793f4c-bc84-4451-a0dc-a025556e36c2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:59.274058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ac33a-ad5b-4feb-aa61-73fed4d94435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:59.274636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ac33a-ad5b-4feb-aa61-73fed4d94435 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:59.274918 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ac33a-ad5b-4feb-aa61-73fed4d94435 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:59.275195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ac33a-ad5b-4feb-aa61-73fed4d94435 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:59.276382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ac33a-ad5b-4feb-aa61-73fed4d94435 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:59.277880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ac33a-ad5b-4feb-aa61-73fed4d94435 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:59.279567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 560/1133] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:31:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:31:59.285762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6436266-760f-4bf5-a935-567246b7b756 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:31:59.358374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:31:59.392367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6436266-760f-4bf5-a935-567246b7b756 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:31:59.427822 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907828bf-2d0e-4b33-b22f-52424a2b48c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:31:59.428464 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907828bf-2d0e-4b33-b22f-52424a2b48c7 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:31:59.428743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907828bf-2d0e-4b33-b22f-52424a2b48c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:31:59.429016 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907828bf-2d0e-4b33-b22f-52424a2b48c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:31:59.429737 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907828bf-2d0e-4b33-b22f-52424a2b48c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:31:59.431177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907828bf-2d0e-4b33-b22f-52424a2b48c7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:31:59.460613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 574/1134] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 3 10:31:59 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:32:01.605360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f2032-cde0-48e3-855f-0f4e5fa0632c None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:01.605934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f2032-cde0-48e3-855f-0f4e5fa0632c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:01.606027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f2032-cde0-48e3-855f-0f4e5fa0632c None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:01.607586 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 561/1135] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:32:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:32:01.613659 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327174fc-853a-4000-8d1e-5089210fd12e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:01.614198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327174fc-853a-4000-8d1e-5089210fd12e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:01.614392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327174fc-853a-4000-8d1e-5089210fd12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:01.633517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-327174fc-853a-4000-8d1e-5089210fd12e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:32:01.634717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-327174fc-853a-4000-8d1e-5089210fd12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:32:01.718810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 575/1136] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:32:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:32:01.744282 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f2032-cde0-48e3-855f-0f4e5fa0632c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:01.832748 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:01.865353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f2032-cde0-48e3-855f-0f4e5fa0632c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:01.900387 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7363ca-714d-48df-ab79-d731b33c613a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:01.901040 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7363ca-714d-48df-ab79-d731b33c613a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:01.901320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7363ca-714d-48df-ab79-d731b33c613a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:01.901625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7363ca-714d-48df-ab79-d731b33c613a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:01.902799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7363ca-714d-48df-ab79-d731b33c613a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:01.904232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7363ca-714d-48df-ab79-d731b33c613a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:01.905893 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 562/1137] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:32:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:01.911846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327174fc-853a-4000-8d1e-5089210fd12e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:01.969278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:02.020297 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327174fc-853a-4000-8d1e-5089210fd12e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:02.055721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdfe1d9-7b8b-4868-b66c-9c5fe46b8dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:02.056343 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfe1d9-7b8b-4868-b66c-9c5fe46b8dd9 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:02.056671 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfe1d9-7b8b-4868-b66c-9c5fe46b8dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:02.056959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfe1d9-7b8b-4868-b66c-9c5fe46b8dd9 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:02.057687 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdfe1d9-7b8b-4868-b66c-9c5fe46b8dd9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:02.059109 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdfe1d9-7b8b-4868-b66c-9c5fe46b8dd9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:02.082492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 576/1138] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:32:01 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:32:04.188959 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6a239-501a-4ba6-bc22-a0897f0769ee None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:04.189551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6a239-501a-4ba6-bc22-a0897f0769ee None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:04.189551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6a239-501a-4ba6-bc22-a0897f0769ee None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:04.190905 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 563/1139] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:32:04.196079 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3c28f-baf4-410c-a833-d2e44858b154 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:04.196510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3c28f-baf4-410c-a833-d2e44858b154 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:04.196631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3c28f-baf4-410c-a833-d2e44858b154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:04.214596 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2ea3c28f-baf4-410c-a833-d2e44858b154 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:32:04.216056 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2ea3c28f-baf4-410c-a833-d2e44858b154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:32:04.282236 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 577/1140] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:32:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:32:04.305142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6a239-501a-4ba6-bc22-a0897f0769ee None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:04.403260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6a239-501a-4ba6-bc22-a0897f0769ee None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:04.435782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170bf57-ccc8-4c30-8576-91077ab953ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:04.436419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170bf57-ccc8-4c30-8576-91077ab953ca None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:04.436695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170bf57-ccc8-4c30-8576-91077ab953ca None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:04.436965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170bf57-ccc8-4c30-8576-91077ab953ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:04.444924 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:04.447978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170bf57-ccc8-4c30-8576-91077ab953ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-3170bf57-ccc8-4c30-8576-91077ab953ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:32:04.452948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:32:04.454759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:32:04.455874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:32:04.455874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:32:04.455874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:32:04.455874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:32:04.455874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:32:04.455874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:32:04.455874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:32:04.455874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:32:04.455874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:32:04.455874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:32:04.455874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:32:04.455874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:32:04.455874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 564/1141] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:32:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:32:04.461038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3c28f-baf4-410c-a833-d2e44858b154 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:04.499523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:04.557896 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3c28f-baf4-410c-a833-d2e44858b154 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:04.590979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f732a86-b506-47d0-9f46-f74439e64407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:04.591594 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f732a86-b506-47d0-9f46-f74439e64407 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:04.591875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f732a86-b506-47d0-9f46-f74439e64407 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:04.592151 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f732a86-b506-47d0-9f46-f74439e64407 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:04.592863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f732a86-b506-47d0-9f46-f74439e64407 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:32:04.593168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f732a86-b506-47d0-9f46-f74439e64407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:04.594546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f732a86-b506-47d0-9f46-f74439e64407 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:04.602515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 578/1142] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:32:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:04.608356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170bf57-ccc8-4c30-8576-91077ab953ca None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:04.706614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170bf57-ccc8-4c30-8576-91077ab953ca None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:04.743703 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca0252-4112-402c-98f7-04753cbe645f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:04.744350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca0252-4112-402c-98f7-04753cbe645f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:04.744634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca0252-4112-402c-98f7-04753cbe645f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:04.744918 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca0252-4112-402c-98f7-04753cbe645f None admin] PATH_INFO: `/v3/users/af8008ae0429476c8ed32ddd798edf40` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:04.759570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca0252-4112-402c-98f7-04753cbe645f None admin] RBAC: Authorizing `identity:get_user(user_id=af8008ae0429476c8ed32ddd798edf40)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:04.761006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca0252-4112-402c-98f7-04753cbe645f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:04.762704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 565/1143] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:32:04 2022] GET /identity/v3/users/af8008ae0429476c8ed32ddd798edf40 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:04.768535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f732a86-b506-47d0-9f46-f74439e64407 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:04.864136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:04.867307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f732a86-b506-47d0-9f46-f74439e64407 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:04.899606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8be51d-335d-4b81-87c8-5a3599ac8f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:04.900221 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8be51d-335d-4b81-87c8-5a3599ac8f47 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:04.900503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8be51d-335d-4b81-87c8-5a3599ac8f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:04.900773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8be51d-335d-4b81-87c8-5a3599ac8f47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:04.976664 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8be51d-335d-4b81-87c8-5a3599ac8f47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:04.978156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8be51d-335d-4b81-87c8-5a3599ac8f47 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:05.078661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:05.082836 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 579/1144] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:32:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 10:32:05.088578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca0252-4112-402c-98f7-04753cbe645f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:05.146282 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:05.186962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca0252-4112-402c-98f7-04753cbe645f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:05.220152 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4a6ba-5764-4a93-a546-e4d11e7e0106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:05.220822 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4a6ba-5764-4a93-a546-e4d11e7e0106 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:05.221067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4a6ba-5764-4a93-a546-e4d11e7e0106 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:05.221343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4a6ba-5764-4a93-a546-e4d11e7e0106 None admin] PATH_INFO: `/v3/projects/9b2a101c98b94caa9d344de050dda02d` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:05.223050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4a6ba-5764-4a93-a546-e4d11e7e0106 None admin] RBAC: Authorizing `identity:get_project(project_id=9b2a101c98b94caa9d344de050dda02d)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:05.224486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4a6ba-5764-4a93-a546-e4d11e7e0106 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:05.226230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 566/1145] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:32:05 2022] GET /identity/v3/projects/9b2a101c98b94caa9d344de050dda02d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:05.232721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8be51d-335d-4b81-87c8-5a3599ac8f47 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:05.327419 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8be51d-335d-4b81-87c8-5a3599ac8f47 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:05.361442 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a642bbe-b548-41f7-bfc3-d19f4e865986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:05.362085 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a642bbe-b548-41f7-bfc3-d19f4e865986 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:05.362367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a642bbe-b548-41f7-bfc3-d19f4e865986 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:05.362644 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a642bbe-b548-41f7-bfc3-d19f4e865986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:05.363430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a642bbe-b548-41f7-bfc3-d19f4e865986 None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=9b2a101c98b94caa9d344de050dda02d, user.id=af8008ae0429476c8ed32ddd798edf40) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:32:05.363738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a642bbe-b548-41f7-bfc3-d19f4e865986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:05.365152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a642bbe-b548-41f7-bfc3-d19f4e865986 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:05.370525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 580/1146] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:32:05 2022] GET /identity/v3/role_assignments?user.id=af8008ae0429476c8ed32ddd798edf40&scope.project.id=9b2a101c98b94caa9d344de050dda02d&role.id=01714797cab143e1a7a150c068fa9315 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:07.351176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2bf26-bd94-4cc8-8be4-bb77655f4835 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:07.351705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2bf26-bd94-4cc8-8be4-bb77655f4835 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:07.351705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2bf26-bd94-4cc8-8be4-bb77655f4835 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:07.353077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 567/1147] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:32:07.358356 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa3678-ebd4-4eb0-a8c7-9edb73a66c28 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:07.358806 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa3678-ebd4-4eb0-a8c7-9edb73a66c28 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:07.358909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa3678-ebd4-4eb0-a8c7-9edb73a66c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:07.377165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f7aa3678-ebd4-4eb0-a8c7-9edb73a66c28 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:32:07.378391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f7aa3678-ebd4-4eb0-a8c7-9edb73a66c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:32:07.435550 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:07.443884 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 581/1148] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:32:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:32:07.466423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2bf26-bd94-4cc8-8be4-bb77655f4835 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:07.565258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2bf26-bd94-4cc8-8be4-bb77655f4835 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:07.590864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:07.600439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f3b8e-b358-4d9d-8e34-9ce482bc4917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:07.601075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f3b8e-b358-4d9d-8e34-9ce482bc4917 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:07.601401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f3b8e-b358-4d9d-8e34-9ce482bc4917 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:07.601662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f3b8e-b358-4d9d-8e34-9ce482bc4917 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:07.611507 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f3b8e-b358-4d9d-8e34-9ce482bc4917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-196f3b8e-b358-4d9d-8e34-9ce482bc4917 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:32:07.616383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:32:07.617506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:32:07.618388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:32:07.618388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:32:07.618388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:32:07.618388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:32:07.618388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:32:07.618388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:32:07.618388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:32:07.618388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:32:07.618388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:32:07.618388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:32:07.618388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:32:07.618388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:32:07.618388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 568/1149] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:32:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:32:07.624006 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa3678-ebd4-4eb0-a8c7-9edb73a66c28 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:07.724663 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa3678-ebd4-4eb0-a8c7-9edb73a66c28 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:07.757259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5ef60-f547-4d10-9473-2e57901f105e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:07.757914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5ef60-f547-4d10-9473-2e57901f105e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:07.758200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5ef60-f547-4d10-9473-2e57901f105e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:07.758478 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5ef60-f547-4d10-9473-2e57901f105e None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:07.759196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5ef60-f547-4d10-9473-2e57901f105e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:32:07.759504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5ef60-f547-4d10-9473-2e57901f105e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:07.760869 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5ef60-f547-4d10-9473-2e57901f105e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:07.769268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 582/1150] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:32:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:07.776748 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f3b8e-b358-4d9d-8e34-9ce482bc4917 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:07.879937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f3b8e-b358-4d9d-8e34-9ce482bc4917 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:07.913240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2717f-2037-49a3-9952-705a57a643e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:07.913923 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2717f-2037-49a3-9952-705a57a643e2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:07.914209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2717f-2037-49a3-9952-705a57a643e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:07.914489 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2717f-2037-49a3-9952-705a57a643e2 None admin] PATH_INFO: `/v3/users/af8008ae0429476c8ed32ddd798edf40` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:07.915653 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2717f-2037-49a3-9952-705a57a643e2 None admin] RBAC: Authorizing `identity:get_user(user_id=af8008ae0429476c8ed32ddd798edf40)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:07.917093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2717f-2037-49a3-9952-705a57a643e2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:07.918755 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 569/1151] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:32:07 2022] GET /identity/v3/users/af8008ae0429476c8ed32ddd798edf40 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:07.924738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5ef60-f547-4d10-9473-2e57901f105e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:07.982015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:08.028141 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5ef60-f547-4d10-9473-2e57901f105e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:08.060714 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a19e3b-769a-4f6d-b3ea-7168cdbb39cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:08.061346 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a19e3b-769a-4f6d-b3ea-7168cdbb39cb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:08.061714 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a19e3b-769a-4f6d-b3ea-7168cdbb39cb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:08.061985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a19e3b-769a-4f6d-b3ea-7168cdbb39cb None admin] PATH_INFO: `/v3/projects/9b2a101c98b94caa9d344de050dda02d` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:08.063124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a19e3b-769a-4f6d-b3ea-7168cdbb39cb None admin] RBAC: Authorizing `identity:get_project(project_id=9b2a101c98b94caa9d344de050dda02d)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:08.065059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a19e3b-769a-4f6d-b3ea-7168cdbb39cb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:08.066761 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 583/1152] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:32:07 2022] GET /identity/v3/projects/9b2a101c98b94caa9d344de050dda02d => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:08.072735 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2717f-2037-49a3-9952-705a57a643e2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:08.128558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:08.176380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2717f-2037-49a3-9952-705a57a643e2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:08.212119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34104b37-aea9-435e-801e-68b1acab152e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:08.212787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34104b37-aea9-435e-801e-68b1acab152e None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:08.213066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34104b37-aea9-435e-801e-68b1acab152e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:08.213339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34104b37-aea9-435e-801e-68b1acab152e None admin] PATH_INFO: `/v3/projects/9b2a101c98b94caa9d344de050dda02d/users/af8008ae0429476c8ed32ddd798edf40/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:08.215522 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34104b37-aea9-435e-801e-68b1acab152e None admin] RBAC: Authorizing `identity:create_grant(project_id=9b2a101c98b94caa9d344de050dda02d, user_id=af8008ae0429476c8ed32ddd798edf40, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:08.216947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34104b37-aea9-435e-801e-68b1acab152e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:08.230508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 570/1153] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:32:08 2022] PUT /identity/v3/projects/9b2a101c98b94caa9d344de050dda02d/users/af8008ae0429476c8ed32ddd798edf40/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:32:10.192760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819352ed-cbb7-43c1-ad08-4ead6577d762 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:10.193287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819352ed-cbb7-43c1-ad08-4ead6577d762 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:10.193287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819352ed-cbb7-43c1-ad08-4ead6577d762 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:10.194687 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 584/1154] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:32:10.199815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009dfb7-a585-403c-975b-75e834563e57 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:10.200250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009dfb7-a585-403c-975b-75e834563e57 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:10.200250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009dfb7-a585-403c-975b-75e834563e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:10.218959 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f009dfb7-a585-403c-975b-75e834563e57 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:32:10.220675 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f009dfb7-a585-403c-975b-75e834563e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:32:10.286503 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 571/1155] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:32:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:32:10.309656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819352ed-cbb7-43c1-ad08-4ead6577d762 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:10.415565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819352ed-cbb7-43c1-ad08-4ead6577d762 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:10.438546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:10.449373 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02150af-9123-41d5-8485-ceee9629b02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:10.449993 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02150af-9123-41d5-8485-ceee9629b02f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:10.450283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02150af-9123-41d5-8485-ceee9629b02f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:10.450554 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02150af-9123-41d5-8485-ceee9629b02f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:10.460867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02150af-9123-41d5-8485-ceee9629b02f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-e02150af-9123-41d5-8485-ceee9629b02f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:32:10.466100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:32:10.467168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:32:10.468192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:32:10.468192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:32:10.468192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:32:10.468192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:32:10.468192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:32:10.468192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:32:10.468192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:32:10.468192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:32:10.468192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:32:10.468192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:32:10.468192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:32:10.468192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:32:10.468192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 585/1156] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:32:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:32:10.473937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009dfb7-a585-403c-975b-75e834563e57 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:10.517940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:10.570351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009dfb7-a585-403c-975b-75e834563e57 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:10.604387 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee45849-924d-471d-83a4-6287d4c13b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:10.605015 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee45849-924d-471d-83a4-6287d4c13b94 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:10.605301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee45849-924d-471d-83a4-6287d4c13b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:10.605613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee45849-924d-471d-83a4-6287d4c13b94 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:10.606338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee45849-924d-471d-83a4-6287d4c13b94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:32:10.606644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee45849-924d-471d-83a4-6287d4c13b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:10.608005 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee45849-924d-471d-83a4-6287d4c13b94 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:10.616000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 572/1157] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:32:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:10.621821 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02150af-9123-41d5-8485-ceee9629b02f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:10.721116 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02150af-9123-41d5-8485-ceee9629b02f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:10.753587 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616cbf0a-8d9f-4642-aade-99c2e76752c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:10.754227 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cbf0a-8d9f-4642-aade-99c2e76752c8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:10.754507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cbf0a-8d9f-4642-aade-99c2e76752c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:10.754782 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cbf0a-8d9f-4642-aade-99c2e76752c8 None admin] PATH_INFO: `/v3/users/af8008ae0429476c8ed32ddd798edf40` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:10.755947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616cbf0a-8d9f-4642-aade-99c2e76752c8 None admin] RBAC: Authorizing `identity:get_user(user_id=af8008ae0429476c8ed32ddd798edf40)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:10.757370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616cbf0a-8d9f-4642-aade-99c2e76752c8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:10.759013 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 586/1158] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:32:10 2022] GET /identity/v3/users/af8008ae0429476c8ed32ddd798edf40 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:10.764816 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee45849-924d-471d-83a4-6287d4c13b94 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:10.863127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee45849-924d-471d-83a4-6287d4c13b94 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:10.869768 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:10.896559 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be6620-b020-4c73-9068-9e4ce9176f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:10.897171 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be6620-b020-4c73-9068-9e4ce9176f49 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:10.897449 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be6620-b020-4c73-9068-9e4ce9176f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:10.897753 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be6620-b020-4c73-9068-9e4ce9176f49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:10.962072 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be6620-b020-4c73-9068-9e4ce9176f49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:10.963511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be6620-b020-4c73-9068-9e4ce9176f49 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:11.056560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 573/1159] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:32:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 10:32:11.062442 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616cbf0a-8d9f-4642-aade-99c2e76752c8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:11.113932 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:11.160480 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616cbf0a-8d9f-4642-aade-99c2e76752c8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:11.193646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43f24b-f49d-4dcb-9b29-af493883fd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:11.194273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43f24b-f49d-4dcb-9b29-af493883fd7a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:11.194541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43f24b-f49d-4dcb-9b29-af493883fd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:11.194810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43f24b-f49d-4dcb-9b29-af493883fd7a None admin] PATH_INFO: `/v3/projects/9b2a101c98b94caa9d344de050dda02d` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:11.195942 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43f24b-f49d-4dcb-9b29-af493883fd7a None admin] RBAC: Authorizing `identity:get_project(project_id=9b2a101c98b94caa9d344de050dda02d)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:11.197349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43f24b-f49d-4dcb-9b29-af493883fd7a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:11.199037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 587/1160] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:32:11 2022] GET /identity/v3/projects/9b2a101c98b94caa9d344de050dda02d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:11.205895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be6620-b020-4c73-9068-9e4ce9176f49 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:11.213834 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:11.310054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be6620-b020-4c73-9068-9e4ce9176f49 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:11.343522 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6684f60-5e9d-4d2d-9e3c-e79956d8aeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:11.344131 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6684f60-5e9d-4d2d-9e3c-e79956d8aeff None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:11.344410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6684f60-5e9d-4d2d-9e3c-e79956d8aeff None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:11.344706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6684f60-5e9d-4d2d-9e3c-e79956d8aeff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:11.345508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6684f60-5e9d-4d2d-9e3c-e79956d8aeff None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=9b2a101c98b94caa9d344de050dda02d, user.id=af8008ae0429476c8ed32ddd798edf40) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:32:11.345810 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6684f60-5e9d-4d2d-9e3c-e79956d8aeff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:11.347222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6684f60-5e9d-4d2d-9e3c-e79956d8aeff None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:11.352806 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 574/1161] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:32:11 2022] GET /identity/v3/role_assignments?user.id=af8008ae0429476c8ed32ddd798edf40&scope.project.id=9b2a101c98b94caa9d344de050dda02d&role.id=01714797cab143e1a7a150c068fa9315 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:13.388350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8288e1-fd9c-4784-bca5-ce551ffa6ea2 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:13.388888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8288e1-fd9c-4784-bca5-ce551ffa6ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:13.388888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8288e1-fd9c-4784-bca5-ce551ffa6ea2 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:13.390270 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 588/1162] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:32:13.395411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937006a0-8e5c-49d2-b326-af92cc8810a0 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:13.395836 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937006a0-8e5c-49d2-b326-af92cc8810a0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:13.395969 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937006a0-8e5c-49d2-b326-af92cc8810a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:13.414320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-937006a0-8e5c-49d2-b326-af92cc8810a0 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:32:13.415536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-937006a0-8e5c-49d2-b326-af92cc8810a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:32:13.477384 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:13.482967 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 575/1163] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:32:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:32:13.504950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8288e1-fd9c-4784-bca5-ce551ffa6ea2 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:13.605237 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8288e1-fd9c-4784-bca5-ce551ffa6ea2 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:13.624690 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:13.638952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81de10-0f52-4935-befe-eaf823f0f292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:13.639505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81de10-0f52-4935-befe-eaf823f0f292 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:13.639786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81de10-0f52-4935-befe-eaf823f0f292 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:13.640062 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81de10-0f52-4935-befe-eaf823f0f292 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:13.714529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81de10-0f52-4935-befe-eaf823f0f292 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:13.716241 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81de10-0f52-4935-befe-eaf823f0f292 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:13.820101 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 589/1164] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:32:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 10:32:13.826359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937006a0-8e5c-49d2-b326-af92cc8810a0 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:13.920711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937006a0-8e5c-49d2-b326-af92cc8810a0 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:13.953075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f4eed-ba01-4013-be46-fd7a50b1aab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:13.953740 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f4eed-ba01-4013-be46-fd7a50b1aab1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:13.954588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f4eed-ba01-4013-be46-fd7a50b1aab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:13.954860 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f4eed-ba01-4013-be46-fd7a50b1aab1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:13.961333 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f4eed-ba01-4013-be46-fd7a50b1aab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:13.962799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f4eed-ba01-4013-be46-fd7a50b1aab1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-923f4eed-ba01-4013-be46-fd7a50b1aab1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:32:13.974310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:32:13.975533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:32:13.976619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:32:13.976619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:32:13.976619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:32:13.976619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:32:13.976619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:32:13.976619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:32:13.976619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:32:13.976619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:32:13.976619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:32:13.976619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:32:13.976619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 03 10:32:13.976619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:32:13.976619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 03 10:32:13.976619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:32:13.976619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 576/1165] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 3 10:32:13 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:32:13.981702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81de10-0f52-4935-befe-eaf823f0f292 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:14.002775 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:14.078057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81de10-0f52-4935-befe-eaf823f0f292 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:14.111446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336681d4-a803-4da7-91fa-b627ac076839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:14.112063 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336681d4-a803-4da7-91fa-b627ac076839 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:14.112345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336681d4-a803-4da7-91fa-b627ac076839 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:14.112618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336681d4-a803-4da7-91fa-b627ac076839 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:14.113307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336681d4-a803-4da7-91fa-b627ac076839 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:32:14.113641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336681d4-a803-4da7-91fa-b627ac076839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:14.115003 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336681d4-a803-4da7-91fa-b627ac076839 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:14.122015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 590/1166] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Aug 3 10:32:13 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:16.122083 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecdd79-6aba-4210-a99f-ea3ca27966be None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:16.122613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecdd79-6aba-4210-a99f-ea3ca27966be None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:16.122613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecdd79-6aba-4210-a99f-ea3ca27966be None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:16.123998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 577/1167] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:32:16.129882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664d7f0-9ecb-4282-8d95-13605fd5f8cc None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:16.130321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664d7f0-9ecb-4282-8d95-13605fd5f8cc None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:16.130423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664d7f0-9ecb-4282-8d95-13605fd5f8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:16.150364 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0664d7f0-9ecb-4282-8d95-13605fd5f8cc None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:32:16.151570 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0664d7f0-9ecb-4282-8d95-13605fd5f8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:32:16.225488 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 591/1168] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:32:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:32:16.247679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ecdd79-6aba-4210-a99f-ea3ca27966be None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:16.310965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:16.354590 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ecdd79-6aba-4210-a99f-ea3ca27966be None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:16.388983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eeb0f6-bd4d-4987-aaf4-3a8f28917940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:16.389559 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eeb0f6-bd4d-4987-aaf4-3a8f28917940 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:16.389843 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eeb0f6-bd4d-4987-aaf4-3a8f28917940 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:16.390116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eeb0f6-bd4d-4987-aaf4-3a8f28917940 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:16.390821 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eeb0f6-bd4d-4987-aaf4-3a8f28917940 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:16.392774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eeb0f6-bd4d-4987-aaf4-3a8f28917940 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:16.411272 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 578/1169] 10.222.0.47 () {60 vars in 1138 bytes} [Wed Aug 3 10:32:16 2022] POST /identity/v3/domains => generated 267 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:32:18.467321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca77e86-6a17-413f-9200-94d4f302610e None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:18.467851 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca77e86-6a17-413f-9200-94d4f302610e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:18.467851 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca77e86-6a17-413f-9200-94d4f302610e None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:18.469227 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 592/1170] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:32:18.475053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57491e42-cf0e-4c70-956f-bc30e194e298 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:18.475452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57491e42-cf0e-4c70-956f-bc30e194e298 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:18.475622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57491e42-cf0e-4c70-956f-bc30e194e298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:18.493952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-57491e42-cf0e-4c70-956f-bc30e194e298 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:32:18.495155 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-57491e42-cf0e-4c70-956f-bc30e194e298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:32:18.561710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 579/1171] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:32:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:32:18.584626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca77e86-6a17-413f-9200-94d4f302610e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:18.601862 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:18.693402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca77e86-6a17-413f-9200-94d4f302610e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:18.730478 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0313b-97c5-49d5-b9ed-adaf8914c30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:18.731063 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0313b-97c5-49d5-b9ed-adaf8914c30a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:18.731350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0313b-97c5-49d5-b9ed-adaf8914c30a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:18.731621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0313b-97c5-49d5-b9ed-adaf8914c30a None admin] PATH_INFO: `/v3/domains/3973efaea67041a78dec72c012c0b368` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:18.748191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0313b-97c5-49d5-b9ed-adaf8914c30a None admin] RBAC: Authorizing `identity:get_domain(domain_id=3973efaea67041a78dec72c012c0b368)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:18.749709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0313b-97c5-49d5-b9ed-adaf8914c30a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:18.752017 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 593/1172] 10.222.0.47 () {56 vars in 1183 bytes} [Wed Aug 3 10:32:18 2022] GET /identity/v3/domains/3973efaea67041a78dec72c012c0b368 => generated 267 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:18.760007 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57491e42-cf0e-4c70-956f-bc30e194e298 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:18.811546 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:18.864220 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57491e42-cf0e-4c70-956f-bc30e194e298 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:18.897600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f025e7-477f-446a-9be8-2be82936dbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:18.898230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f025e7-477f-446a-9be8-2be82936dbe0 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:18.898513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f025e7-477f-446a-9be8-2be82936dbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:18.898790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f025e7-477f-446a-9be8-2be82936dbe0 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:18.899471 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f025e7-477f-446a-9be8-2be82936dbe0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:18.900908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f025e7-477f-446a-9be8-2be82936dbe0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:18.927030 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 580/1173] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 3 10:32:18 2022] POST /identity/v3/projects => generated 361 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:32:20.972745 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e70cf-280a-446e-9b01-ac4fc670dbc8 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:20.973311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e70cf-280a-446e-9b01-ac4fc670dbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:20.973311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e70cf-280a-446e-9b01-ac4fc670dbc8 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:20.974719 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 594/1174] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:32:20.979951 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fcff7-a9a1-4da1-b9d7-2c26fcf8dc57 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:20.980376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fcff7-a9a1-4da1-b9d7-2c26fcf8dc57 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:20.980485 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fcff7-a9a1-4da1-b9d7-2c26fcf8dc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:20.998258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-167fcff7-a9a1-4da1-b9d7-2c26fcf8dc57 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:32:20.999440 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-167fcff7-a9a1-4da1-b9d7-2c26fcf8dc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:32:21.063271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 581/1175] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:32:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:32:21.088280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e70cf-280a-446e-9b01-ac4fc670dbc8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:21.169160 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:21.196871 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e70cf-280a-446e-9b01-ac4fc670dbc8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:21.235588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12f1b5-ca46-4fd2-9f44-d2e7dbde3c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:21.236157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12f1b5-ca46-4fd2-9f44-d2e7dbde3c10 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:21.236437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12f1b5-ca46-4fd2-9f44-d2e7dbde3c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:21.236707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12f1b5-ca46-4fd2-9f44-d2e7dbde3c10 None admin] PATH_INFO: `/v3/domains/3973efaea67041a78dec72c012c0b368` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:21.237886 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12f1b5-ca46-4fd2-9f44-d2e7dbde3c10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3973efaea67041a78dec72c012c0b368)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:21.239299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12f1b5-ca46-4fd2-9f44-d2e7dbde3c10 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:21.240893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 595/1176] 10.222.0.47 () {56 vars in 1183 bytes} [Wed Aug 3 10:32:21 2022] GET /identity/v3/domains/3973efaea67041a78dec72c012c0b368 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:21.246710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fcff7-a9a1-4da1-b9d7-2c26fcf8dc57 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:21.276616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:21.345123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fcff7-a9a1-4da1-b9d7-2c26fcf8dc57 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:21.378339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e6ea8-190b-4c93-90a3-ef05b0550032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:21.379131 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e6ea8-190b-4c93-90a3-ef05b0550032 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:21.379131 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e6ea8-190b-4c93-90a3-ef05b0550032 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:21.379935 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e6ea8-190b-4c93-90a3-ef05b0550032 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:21.380195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e6ea8-190b-4c93-90a3-ef05b0550032 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:21.381818 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e6ea8-190b-4c93-90a3-ef05b0550032 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:21.403807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 582/1177] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:32:21 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:32:23.459116 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c2260-0122-44f9-bebe-7725bb05fbf4 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:23.459678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c2260-0122-44f9-bebe-7725bb05fbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:23.459678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c2260-0122-44f9-bebe-7725bb05fbf4 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:23.461015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 596/1178] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:32:23.466741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755b5fa-2912-45fe-b105-e3c496e25712 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:23.467129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755b5fa-2912-45fe-b105-e3c496e25712 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:23.467307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755b5fa-2912-45fe-b105-e3c496e25712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:23.485792 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d755b5fa-2912-45fe-b105-e3c496e25712 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:32:23.486986 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d755b5fa-2912-45fe-b105-e3c496e25712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:32:23.551156 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 583/1179] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:32:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:32:23.574931 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c2260-0122-44f9-bebe-7725bb05fbf4 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:23.673835 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c2260-0122-44f9-bebe-7725bb05fbf4 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:23.708301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc103f7-bd07-4891-951b-b88067e4b44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:23.708939 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc103f7-bd07-4891-951b-b88067e4b44d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:23.709272 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc103f7-bd07-4891-951b-b88067e4b44d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:23.709530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc103f7-bd07-4891-951b-b88067e4b44d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:23.719699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc103f7-bd07-4891-951b-b88067e4b44d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-8fc103f7-bd07-4891-951b-b88067e4b44d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:32:23.724515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:32:23.725738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:32:23.726826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:32:23.726826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:32:23.726826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:32:23.726826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:32:23.726826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:32:23.726826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:32:23.726826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:32:23.726826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:32:23.726826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:32:23.726826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:32:23.726826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:32:23.726826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:32:23.726826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 597/1180] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:32:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:32:23.731931 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755b5fa-2912-45fe-b105-e3c496e25712 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:23.749883 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:23.837231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755b5fa-2912-45fe-b105-e3c496e25712 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:23.870048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c02d1-6f03-4577-82a1-1ac9a16f7270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:23.870672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c02d1-6f03-4577-82a1-1ac9a16f7270 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:23.870955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c02d1-6f03-4577-82a1-1ac9a16f7270 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:23.871230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c02d1-6f03-4577-82a1-1ac9a16f7270 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:23.871947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c02d1-6f03-4577-82a1-1ac9a16f7270 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:32:23.872252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c02d1-6f03-4577-82a1-1ac9a16f7270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:23.873721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c02d1-6f03-4577-82a1-1ac9a16f7270 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:23.882066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 584/1181] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:32:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:23.887917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc103f7-bd07-4891-951b-b88067e4b44d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:23.891898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:23.984566 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc103f7-bd07-4891-951b-b88067e4b44d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:24.018001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17316e7e-9da5-431c-a7b4-ed67e84b9dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:24.018638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17316e7e-9da5-431c-a7b4-ed67e84b9dba None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:24.018920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17316e7e-9da5-431c-a7b4-ed67e84b9dba None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:24.019192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17316e7e-9da5-431c-a7b4-ed67e84b9dba None admin] PATH_INFO: `/v3/users/45a3f35cdb3e419980e3e3e73de2f6a9` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:24.032076 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17316e7e-9da5-431c-a7b4-ed67e84b9dba None admin] RBAC: Authorizing `identity:get_user(user_id=45a3f35cdb3e419980e3e3e73de2f6a9)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:24.033535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17316e7e-9da5-431c-a7b4-ed67e84b9dba None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:24.035203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 598/1182] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:32:23 2022] GET /identity/v3/users/45a3f35cdb3e419980e3e3e73de2f6a9 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:24.041229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c02d1-6f03-4577-82a1-1ac9a16f7270 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:24.124885 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:24.144437 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c02d1-6f03-4577-82a1-1ac9a16f7270 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:24.176858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6436e-9a6b-4107-aa9e-df3eb2e82206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:24.177504 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6436e-9a6b-4107-aa9e-df3eb2e82206 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:24.177789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6436e-9a6b-4107-aa9e-df3eb2e82206 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:24.178067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6436e-9a6b-4107-aa9e-df3eb2e82206 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:24.252438 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6436e-9a6b-4107-aa9e-df3eb2e82206 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:24.253815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6436e-9a6b-4107-aa9e-df3eb2e82206 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:24.330151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:24.350508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 585/1183] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:32:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 10:32:24.356401 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17316e7e-9da5-431c-a7b4-ed67e84b9dba None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:24.414112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:24.455961 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17316e7e-9da5-431c-a7b4-ed67e84b9dba None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:24.493271 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67303be-f587-47aa-9353-8e2ab7ed1c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:24.494016 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67303be-f587-47aa-9353-8e2ab7ed1c62 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:24.494262 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67303be-f587-47aa-9353-8e2ab7ed1c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:24.494548 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67303be-f587-47aa-9353-8e2ab7ed1c62 None admin] PATH_INFO: `/v3/projects/5cd94d14afbb4e4f8acb86e89e226922` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:24.495845 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67303be-f587-47aa-9353-8e2ab7ed1c62 None admin] RBAC: Authorizing `identity:get_project(project_id=5cd94d14afbb4e4f8acb86e89e226922)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:24.497354 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67303be-f587-47aa-9353-8e2ab7ed1c62 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:24.499183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 599/1184] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:32:24 2022] GET /identity/v3/projects/5cd94d14afbb4e4f8acb86e89e226922 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:24.505964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6436e-9a6b-4107-aa9e-df3eb2e82206 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:24.611327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6436e-9a6b-4107-aa9e-df3eb2e82206 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:24.645141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9964ca-a87a-406e-936a-4c149f9729fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:24.645814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9964ca-a87a-406e-936a-4c149f9729fa None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:24.646135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9964ca-a87a-406e-936a-4c149f9729fa None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:24.646358 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9964ca-a87a-406e-936a-4c149f9729fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:24.647146 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9964ca-a87a-406e-936a-4c149f9729fa None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=5cd94d14afbb4e4f8acb86e89e226922, user.id=45a3f35cdb3e419980e3e3e73de2f6a9) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:32:24.647456 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9964ca-a87a-406e-936a-4c149f9729fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:24.648882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9964ca-a87a-406e-936a-4c149f9729fa None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:24.654421 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 586/1185] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:32:24 2022] GET /identity/v3/role_assignments?user.id=45a3f35cdb3e419980e3e3e73de2f6a9&scope.project.id=5cd94d14afbb4e4f8acb86e89e226922&role.id=01714797cab143e1a7a150c068fa9315 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:26.615721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3fb0a-e9bb-43bd-9401-5ee17da08552 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:26.616252 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3fb0a-e9bb-43bd-9401-5ee17da08552 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:26.616252 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3fb0a-e9bb-43bd-9401-5ee17da08552 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:26.618011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 600/1186] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:32:26.623229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f4d6e-5d83-4c67-bc82-403485006a6d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:26.623669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f4d6e-5d83-4c67-bc82-403485006a6d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:26.623769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f4d6e-5d83-4c67-bc82-403485006a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:26.641312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c29f4d6e-5d83-4c67-bc82-403485006a6d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:32:26.642509 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c29f4d6e-5d83-4c67-bc82-403485006a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:32:26.681156 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:26.707551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 587/1187] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:32:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:32:26.730511 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3fb0a-e9bb-43bd-9401-5ee17da08552 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:26.832546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3fb0a-e9bb-43bd-9401-5ee17da08552 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:26.863340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:26.871734 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe6b69-ef30-4703-bb65-1bcc211314d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:26.872309 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe6b69-ef30-4703-bb65-1bcc211314d6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:26.872595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe6b69-ef30-4703-bb65-1bcc211314d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:26.872873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe6b69-ef30-4703-bb65-1bcc211314d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:26.882907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe6b69-ef30-4703-bb65-1bcc211314d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-39fe6b69-ef30-4703-bb65-1bcc211314d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:32:26.887781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:32:26.888866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:32:26.889764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:32:26.889764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:32:26.889764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:32:26.889764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:32:26.889764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:32:26.889764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:32:26.889764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:32:26.889764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:32:26.889764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:32:26.889764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:32:26.889764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:32:26.889764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:32:26.889764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 601/1188] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:32:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:32:26.895562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f4d6e-5d83-4c67-bc82-403485006a6d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:26.996329 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f4d6e-5d83-4c67-bc82-403485006a6d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:27.030080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673490e-7782-4d25-af0e-f0fa49ef24d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:27.030709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673490e-7782-4d25-af0e-f0fa49ef24d3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:27.030995 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673490e-7782-4d25-af0e-f0fa49ef24d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:27.031271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673490e-7782-4d25-af0e-f0fa49ef24d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:27.032574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673490e-7782-4d25-af0e-f0fa49ef24d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:32:27.032881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673490e-7782-4d25-af0e-f0fa49ef24d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:27.034296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673490e-7782-4d25-af0e-f0fa49ef24d3 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:27.042272 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 588/1189] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:32:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:27.048110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe6b69-ef30-4703-bb65-1bcc211314d6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:27.146257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe6b69-ef30-4703-bb65-1bcc211314d6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:27.179332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa043e5-ee6c-49dc-bc65-ff232f43de0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:27.179971 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa043e5-ee6c-49dc-bc65-ff232f43de0b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:27.180254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa043e5-ee6c-49dc-bc65-ff232f43de0b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:27.180530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa043e5-ee6c-49dc-bc65-ff232f43de0b None admin] PATH_INFO: `/v3/users/45a3f35cdb3e419980e3e3e73de2f6a9` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:27.181724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa043e5-ee6c-49dc-bc65-ff232f43de0b None admin] RBAC: Authorizing `identity:get_user(user_id=45a3f35cdb3e419980e3e3e73de2f6a9)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:27.183150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa043e5-ee6c-49dc-bc65-ff232f43de0b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:27.184764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 602/1190] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:32:27 2022] GET /identity/v3/users/45a3f35cdb3e419980e3e3e73de2f6a9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:27.190600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673490e-7782-4d25-af0e-f0fa49ef24d3 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:27.226552 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:27.290199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673490e-7782-4d25-af0e-f0fa49ef24d3 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:27.322491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f438c4-9876-4a43-bbc5-82c034dd422b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:27.323127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f438c4-9876-4a43-bbc5-82c034dd422b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:27.323406 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f438c4-9876-4a43-bbc5-82c034dd422b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:27.323676 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f438c4-9876-4a43-bbc5-82c034dd422b None admin] PATH_INFO: `/v3/projects/5cd94d14afbb4e4f8acb86e89e226922` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:27.324806 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f438c4-9876-4a43-bbc5-82c034dd422b None admin] RBAC: Authorizing `identity:get_project(project_id=5cd94d14afbb4e4f8acb86e89e226922)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:27.326249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f438c4-9876-4a43-bbc5-82c034dd422b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:27.327905 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 589/1191] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:32:27 2022] GET /identity/v3/projects/5cd94d14afbb4e4f8acb86e89e226922 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:27.334545 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa043e5-ee6c-49dc-bc65-ff232f43de0b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:27.392716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:27.435516 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa043e5-ee6c-49dc-bc65-ff232f43de0b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:27.470390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063f04e-4e55-4f5a-ab42-1ad8461be5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:27.471089 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063f04e-4e55-4f5a-ab42-1ad8461be5e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:27.471369 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063f04e-4e55-4f5a-ab42-1ad8461be5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:27.471642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063f04e-4e55-4f5a-ab42-1ad8461be5e7 None admin] PATH_INFO: `/v3/projects/5cd94d14afbb4e4f8acb86e89e226922/users/45a3f35cdb3e419980e3e3e73de2f6a9/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:27.473713 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063f04e-4e55-4f5a-ab42-1ad8461be5e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5cd94d14afbb4e4f8acb86e89e226922, user_id=45a3f35cdb3e419980e3e3e73de2f6a9, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:27.475177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063f04e-4e55-4f5a-ab42-1ad8461be5e7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:27.489553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 603/1192] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:32:27 2022] PUT /identity/v3/projects/5cd94d14afbb4e4f8acb86e89e226922/users/45a3f35cdb3e419980e3e3e73de2f6a9/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:32:29.532621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35918544-556c-408e-85ab-232f5009b8f2 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:29.533195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35918544-556c-408e-85ab-232f5009b8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:29.533195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35918544-556c-408e-85ab-232f5009b8f2 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:29.534577 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 590/1193] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:32:29.540028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c75b9c-c61d-4602-a631-a32bbfd5a081 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:29.540501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c75b9c-c61d-4602-a631-a32bbfd5a081 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:29.540606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c75b9c-c61d-4602-a631-a32bbfd5a081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:29.559141 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-90c75b9c-c61d-4602-a631-a32bbfd5a081 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:32:29.560349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-90c75b9c-c61d-4602-a631-a32bbfd5a081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:32:29.627093 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 604/1194] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:32:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:32:29.650364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35918544-556c-408e-85ab-232f5009b8f2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:29.754104 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35918544-556c-408e-85ab-232f5009b8f2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:29.756470 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:29.790253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424a660-bda7-4e69-820b-aadce7ff7700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:29.790849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424a660-bda7-4e69-820b-aadce7ff7700 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:29.791130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424a660-bda7-4e69-820b-aadce7ff7700 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:29.791402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424a660-bda7-4e69-820b-aadce7ff7700 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:29.808055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424a660-bda7-4e69-820b-aadce7ff7700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-5424a660-bda7-4e69-820b-aadce7ff7700 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:32:29.813048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:32:29.814178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:32:29.815313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:32:29.815313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:32:29.815313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:32:29.815313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:32:29.815313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:32:29.815313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:32:29.815313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:32:29.815313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:32:29.815313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:32:29.815313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:32:29.815313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:32:29.815313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:32:29.815313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 591/1195] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:32:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:32:29.820556 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c75b9c-c61d-4602-a631-a32bbfd5a081 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:29.868405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:29.921753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c75b9c-c61d-4602-a631-a32bbfd5a081 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:29.955865 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349a615-a15a-4bb6-a03b-b4c0f496a14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:29.956490 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349a615-a15a-4bb6-a03b-b4c0f496a14e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:29.956776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349a615-a15a-4bb6-a03b-b4c0f496a14e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:29.957054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349a615-a15a-4bb6-a03b-b4c0f496a14e None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:29.957815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349a615-a15a-4bb6-a03b-b4c0f496a14e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:32:29.958125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349a615-a15a-4bb6-a03b-b4c0f496a14e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:29.959496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349a615-a15a-4bb6-a03b-b4c0f496a14e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:29.967928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 605/1196] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:32:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:29.974442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424a660-bda7-4e69-820b-aadce7ff7700 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:30.073925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424a660-bda7-4e69-820b-aadce7ff7700 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:30.107924 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd86fd-3a73-46db-be9d-0b98b057737a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:30.108573 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd86fd-3a73-46db-be9d-0b98b057737a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:30.108857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd86fd-3a73-46db-be9d-0b98b057737a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:30.109138 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd86fd-3a73-46db-be9d-0b98b057737a None admin] PATH_INFO: `/v3/users/45a3f35cdb3e419980e3e3e73de2f6a9` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:30.110405 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd86fd-3a73-46db-be9d-0b98b057737a None admin] RBAC: Authorizing `identity:get_user(user_id=45a3f35cdb3e419980e3e3e73de2f6a9)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:30.111890 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd86fd-3a73-46db-be9d-0b98b057737a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:30.113550 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 592/1197] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 3 10:32:29 2022] GET /identity/v3/users/45a3f35cdb3e419980e3e3e73de2f6a9 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:30.119481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349a615-a15a-4bb6-a03b-b4c0f496a14e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:30.220122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349a615-a15a-4bb6-a03b-b4c0f496a14e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:30.230669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:30.256175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc16520-e70c-4869-8cd9-597a5ef6a1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:30.256794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc16520-e70c-4869-8cd9-597a5ef6a1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:30.257071 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc16520-e70c-4869-8cd9-597a5ef6a1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:30.257342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc16520-e70c-4869-8cd9-597a5ef6a1c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:30.323489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc16520-e70c-4869-8cd9-597a5ef6a1c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:30.324966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc16520-e70c-4869-8cd9-597a5ef6a1c2 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:30.423061 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 606/1198] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:32:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 10:32:30.429098 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd86fd-3a73-46db-be9d-0b98b057737a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:30.464173 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:30.531890 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd86fd-3a73-46db-be9d-0b98b057737a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:30.564521 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4156f-340f-4e42-b8b5-7dc803c60d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:30.565166 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4156f-340f-4e42-b8b5-7dc803c60d93 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:30.565444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4156f-340f-4e42-b8b5-7dc803c60d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:30.565750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4156f-340f-4e42-b8b5-7dc803c60d93 None admin] PATH_INFO: `/v3/projects/5cd94d14afbb4e4f8acb86e89e226922` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:30.566890 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4156f-340f-4e42-b8b5-7dc803c60d93 None admin] RBAC: Authorizing `identity:get_project(project_id=5cd94d14afbb4e4f8acb86e89e226922)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:30.568308 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4156f-340f-4e42-b8b5-7dc803c60d93 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:30.570001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 593/1199] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 3 10:32:30 2022] GET /identity/v3/projects/5cd94d14afbb4e4f8acb86e89e226922 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:30.576449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc16520-e70c-4869-8cd9-597a5ef6a1c2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:32:30.586494 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:30.675363 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc16520-e70c-4869-8cd9-597a5ef6a1c2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:32:30.709738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d63389-7c52-4fba-87a2-264a3b7f486a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:32:30.710353 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d63389-7c52-4fba-87a2-264a3b7f486a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:32:30.710631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d63389-7c52-4fba-87a2-264a3b7f486a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:32:30.710902 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d63389-7c52-4fba-87a2-264a3b7f486a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:32:30.711684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d63389-7c52-4fba-87a2-264a3b7f486a None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=5cd94d14afbb4e4f8acb86e89e226922, user.id=45a3f35cdb3e419980e3e3e73de2f6a9) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:32:30.711990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d63389-7c52-4fba-87a2-264a3b7f486a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:32:30.713399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d63389-7c52-4fba-87a2-264a3b7f486a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:32:30.719260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 607/1200] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:32:30 2022] GET /identity/v3/role_assignments?user.id=45a3f35cdb3e419980e3e3e73de2f6a9&scope.project.id=5cd94d14afbb4e4f8acb86e89e226922&role.id=01714797cab143e1a7a150c068fa9315 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:32:40.560434 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=50 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:32:40.716669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=64 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:28.599152 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3ae2f-da58-4577-ad62-3b4d98ba4104 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:28.599152 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3ae2f-da58-4577-ad62-3b4d98ba4104 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:28.599152 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3ae2f-da58-4577-ad62-3b4d98ba4104 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:28.599152 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 594/1201] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:33:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:33:28.606467 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21c80d-3eff-4c44-9f5f-40e6c58e79a1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:28.606977 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21c80d-3eff-4c44-9f5f-40e6c58e79a1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:28.606977 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21c80d-3eff-4c44-9f5f-40e6c58e79a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:28.628574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6b21c80d-3eff-4c44-9f5f-40e6c58e79a1 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:33:28.629828 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6b21c80d-3eff-4c44-9f5f-40e6c58e79a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:33:28.717366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 608/1202] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:33:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 10:33:28.744556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3ae2f-da58-4577-ad62-3b4d98ba4104 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:28.892549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3ae2f-da58-4577-ad62-3b4d98ba4104 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:28.973944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435bfeba-4d36-4673-9cdc-3094b2b5b587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:28.974463 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bfeba-4d36-4673-9cdc-3094b2b5b587 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:28.974835 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bfeba-4d36-4673-9cdc-3094b2b5b587 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:28.975064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bfeba-4d36-4673-9cdc-3094b2b5b587 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:28.988352 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435bfeba-4d36-4673-9cdc-3094b2b5b587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:28.990117 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435bfeba-4d36-4673-9cdc-3094b2b5b587 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:29.035231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-435bfeba-4d36-4673-9cdc-3094b2b5b587 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:29.035231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:29.035231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:29.035231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:29.035231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:29.035231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:29.035231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:29.035231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:29.035231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:29.035231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:29.035231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:29.035231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:29.035231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:33:29.035231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:33:29.035231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:29.037064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:29.039027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:33:29.039027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:33:29.039027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:29.039027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:29.051181 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 595/1203] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:29.053590 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21c80d-3eff-4c44-9f5f-40e6c58e79a1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:29.209578 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21c80d-3eff-4c44-9f5f-40e6c58e79a1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:29.269521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:29.279010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e887377-3ba6-4c7a-adae-c7843023d8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:29.279664 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e887377-3ba6-4c7a-adae-c7843023d8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:29.279947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e887377-3ba6-4c7a-adae-c7843023d8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:29.280222 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e887377-3ba6-4c7a-adae-c7843023d8d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:29.280936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e887377-3ba6-4c7a-adae-c7843023d8d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:29.281240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e887377-3ba6-4c7a-adae-c7843023d8d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:29.282686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e887377-3ba6-4c7a-adae-c7843023d8d5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:29.306110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 609/1204] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:33:29.308162 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435bfeba-4d36-4673-9cdc-3094b2b5b587 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:29.407698 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:29.494962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435bfeba-4d36-4673-9cdc-3094b2b5b587 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:29.554377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b728100-2ae0-4b51-8c7a-3fa277aafc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:29.555502 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b728100-2ae0-4b51-8c7a-3fa277aafc53 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:29.555783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b728100-2ae0-4b51-8c7a-3fa277aafc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:29.556072 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b728100-2ae0-4b51-8c7a-3fa277aafc53 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:29.556790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b728100-2ae0-4b51-8c7a-3fa277aafc53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:29.561506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b728100-2ae0-4b51-8c7a-3fa277aafc53 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:29.584397 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 596/1205] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 3 10:33:29 2022] POST /identity/v3/users => generated 251 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:33:31.837185 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6c0ec-ba4f-4cf5-831d-0a9df00eff36 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:31.840998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6c0ec-ba4f-4cf5-831d-0a9df00eff36 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:31.841870 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6c0ec-ba4f-4cf5-831d-0a9df00eff36 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:31.845293 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 610/1206] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:33:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:33:31.854586 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36439a80-2bab-49bf-ae82-606137c4ed4d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:31.855139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36439a80-2bab-49bf-ae82-606137c4ed4d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:31.855139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36439a80-2bab-49bf-ae82-606137c4ed4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:31.876976 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-36439a80-2bab-49bf-ae82-606137c4ed4d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:33:31.878450 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-36439a80-2bab-49bf-ae82-606137c4ed4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:33:31.979045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 597/1207] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:33:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:33:32.009286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6c0ec-ba4f-4cf5-831d-0a9df00eff36 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:32.160808 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6c0ec-ba4f-4cf5-831d-0a9df00eff36 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:32.211783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad18484-ae0f-4a73-bc46-4e54c4d02a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:32.212612 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad18484-ae0f-4a73-bc46-4e54c4d02a10 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:32.213021 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad18484-ae0f-4a73-bc46-4e54c4d02a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:32.213385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad18484-ae0f-4a73-bc46-4e54c4d02a10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:32.234324 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad18484-ae0f-4a73-bc46-4e54c4d02a10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-8ad18484-ae0f-4a73-bc46-4e54c4d02a10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:33:32.243165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:32.245119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:32.246853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:32.246853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:32.246853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:32.246853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:32.246853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:33:32.246853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:33:32.246853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:33:32.246853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:33:32.246853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:33:32.246853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:33:32.246853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:33:32.246853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:33:32.246853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 611/1208] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:33:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:33:32.254253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36439a80-2bab-49bf-ae82-606137c4ed4d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:32.331445 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:32.468047 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36439a80-2bab-49bf-ae82-606137c4ed4d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:32.524614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e33a32-8f1a-473f-b177-3e206359f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:32.525414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e33a32-8f1a-473f-b177-3e206359f709 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:32.525865 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e33a32-8f1a-473f-b177-3e206359f709 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:32.526201 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e33a32-8f1a-473f-b177-3e206359f709 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:32.526943 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e33a32-8f1a-473f-b177-3e206359f709 None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:32.527245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e33a32-8f1a-473f-b177-3e206359f709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:32.528631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e33a32-8f1a-473f-b177-3e206359f709 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:32.538886 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 598/1209] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:33:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:32.548479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad18484-ae0f-4a73-bc46-4e54c4d02a10 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:32.618944 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:32.704840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad18484-ae0f-4a73-bc46-4e54c4d02a10 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:32.761228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef762c7-f442-43fd-b33a-46e3f6870ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:32.762157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef762c7-f442-43fd-b33a-46e3f6870ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:32.762447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef762c7-f442-43fd-b33a-46e3f6870ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:32.762729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef762c7-f442-43fd-b33a-46e3f6870ee9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:32.778787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef762c7-f442-43fd-b33a-46e3f6870ee9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:32.780482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef762c7-f442-43fd-b33a-46e3f6870ee9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:32.803498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-eef762c7-f442-43fd-b33a-46e3f6870ee9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:32.803498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:32.803498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:32.803498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:32.803498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:32.803498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:32.803498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:32.803498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:32.803498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:32.803498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:32.803498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:32.803498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:32.803498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:33:32.803498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:33:32.803498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:32.804646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:32.805789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:33:32.805789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:33:32.805789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:32.805789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:33:32.808930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 612/1210] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:32.810427 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e33a32-8f1a-473f-b177-3e206359f709 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:32.953863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:32.958559 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e33a32-8f1a-473f-b177-3e206359f709 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:33.019021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a9c1e-ac5e-4ec3-9d5b-6e6514420abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:33.020839 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a9c1e-ac5e-4ec3-9d5b-6e6514420abb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:33.021634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a9c1e-ac5e-4ec3-9d5b-6e6514420abb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:33.022262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a9c1e-ac5e-4ec3-9d5b-6e6514420abb None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:33.023987 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a9c1e-ac5e-4ec3-9d5b-6e6514420abb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:33.024714 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a9c1e-ac5e-4ec3-9d5b-6e6514420abb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:33.027816 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a9c1e-ac5e-4ec3-9d5b-6e6514420abb None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:33.057561 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 599/1211] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:33.065203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef762c7-f442-43fd-b33a-46e3f6870ee9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:33.219072 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef762c7-f442-43fd-b33a-46e3f6870ee9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:33.221107 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:33.273799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccbab5-c9bc-4a20-8aed-1c6b924fedde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:33.274857 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccbab5-c9bc-4a20-8aed-1c6b924fedde None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:33.275357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccbab5-c9bc-4a20-8aed-1c6b924fedde None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:33.275855 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccbab5-c9bc-4a20-8aed-1c6b924fedde None admin] PATH_INFO: `/v3/users/placement` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:33.287785 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccbab5-c9bc-4a20-8aed-1c6b924fedde None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:33.289785 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccbab5-c9bc-4a20-8aed-1c6b924fedde None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-e3ccbab5-c9bc-4a20-8aed-1c6b924fedde None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:33:33.310477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:33.312729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:33.314699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:33.314699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:33.314699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:33:33.314699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:33:33.314699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:33:33.314699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:33:33.314699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:33:33.314699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:33:33.314699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 10:33:33.314699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:33:33.314699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 613/1212] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:33.319905 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a9c1e-ac5e-4ec3-9d5b-6e6514420abb None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:33.499736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a9c1e-ac5e-4ec3-9d5b-6e6514420abb None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:33.555750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d23412-135f-4c1d-a0de-087fcadb37d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:33.556508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d23412-135f-4c1d-a0de-087fcadb37d0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:33.556934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d23412-135f-4c1d-a0de-087fcadb37d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:33.557358 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d23412-135f-4c1d-a0de-087fcadb37d0 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:33.558557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d23412-135f-4c1d-a0de-087fcadb37d0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:33.558921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d23412-135f-4c1d-a0de-087fcadb37d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:33.561882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d23412-135f-4c1d-a0de-087fcadb37d0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:33.589425 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 600/1213] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Aug 3 10:33:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:33:33.596449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccbab5-c9bc-4a20-8aed-1c6b924fedde None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:33.773205 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccbab5-c9bc-4a20-8aed-1c6b924fedde None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:33.831342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd24e33-22ae-4dee-9da6-37caf0269504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:33.832324 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd24e33-22ae-4dee-9da6-37caf0269504 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:33.832911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd24e33-22ae-4dee-9da6-37caf0269504 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:33.833415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd24e33-22ae-4dee-9da6-37caf0269504 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:33.883758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:33.944685 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd24e33-22ae-4dee-9da6-37caf0269504 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:33.947634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd24e33-22ae-4dee-9da6-37caf0269504 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:34.111455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 614/1214] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:33:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 10:33:34.120878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d23412-135f-4c1d-a0de-087fcadb37d0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:34.146669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:34.254983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d23412-135f-4c1d-a0de-087fcadb37d0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:34.300202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a51d0-a7aa-48a2-b967-39c7ce1dd82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:34.300893 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a51d0-a7aa-48a2-b967-39c7ce1dd82c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:34.301330 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a51d0-a7aa-48a2-b967-39c7ce1dd82c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:34.301616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a51d0-a7aa-48a2-b967-39c7ce1dd82c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:34.313382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a51d0-a7aa-48a2-b967-39c7ce1dd82c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:34.315027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a51d0-a7aa-48a2-b967-39c7ce1dd82c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:34.331983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-9a6a51d0-a7aa-48a2-b967-39c7ce1dd82c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:34.331983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:34.331983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:34.331983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:34.331983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:34.331983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:34.331983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:34.331983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:34.331983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:34.331983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:34.331983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:34.331983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:34.331983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:33:34.331983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:33:34.331983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:34.333842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:34.335624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:33:34.335624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:33:34.335624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:34.335624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:34.335624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 601/1215] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:34.341547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd24e33-22ae-4dee-9da6-37caf0269504 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:34.458835 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:34.530619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd24e33-22ae-4dee-9da6-37caf0269504 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:34.583619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff4702-aba3-4a0e-b517-62da0daaee79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:34.584391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff4702-aba3-4a0e-b517-62da0daaee79 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:34.584678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff4702-aba3-4a0e-b517-62da0daaee79 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:34.584987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff4702-aba3-4a0e-b517-62da0daaee79 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:34.585726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff4702-aba3-4a0e-b517-62da0daaee79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:34.586098 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff4702-aba3-4a0e-b517-62da0daaee79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:34.587608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff4702-aba3-4a0e-b517-62da0daaee79 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:34.602601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 615/1216] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:34.608701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a51d0-a7aa-48a2-b967-39c7ce1dd82c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:34.721929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:34.769126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a51d0-a7aa-48a2-b967-39c7ce1dd82c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:34.810634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f0727-29f2-45fe-bbb0-de9d48c8f38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:34.811316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f0727-29f2-45fe-bbb0-de9d48c8f38a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:34.811604 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f0727-29f2-45fe-bbb0-de9d48c8f38a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:34.811876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f0727-29f2-45fe-bbb0-de9d48c8f38a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:34.820556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f0727-29f2-45fe-bbb0-de9d48c8f38a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:34.822075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f0727-29f2-45fe-bbb0-de9d48c8f38a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:34.835900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-162f0727-29f2-45fe-bbb0-de9d48c8f38a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:33:34.835900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:34.835900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:34.835900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:34.835900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:34.835900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:34.835900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:34.835900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:34.835900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:34.835900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:34.835900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:34.835900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:34.835900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:33:34.835900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:33:34.835900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:34.837807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:34.838902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:33:34.838902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:33:34.838902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:33:34.838902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:33:34.838902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:33:34.838902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:33:34.838902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:33:34.838902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:34.838902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 602/1217] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:33:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:34.843897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff4702-aba3-4a0e-b517-62da0daaee79 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:35.015025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff4702-aba3-4a0e-b517-62da0daaee79 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:35.053552 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:35.069455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a63310-3552-4468-872c-fc027df750d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:35.070197 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a63310-3552-4468-872c-fc027df750d0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:35.070537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a63310-3552-4468-872c-fc027df750d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:35.070837 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a63310-3552-4468-872c-fc027df750d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:35.071600 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a63310-3552-4468-872c-fc027df750d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:35.071950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a63310-3552-4468-872c-fc027df750d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:35.073428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a63310-3552-4468-872c-fc027df750d0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:35.088910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 616/1218] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:33:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:35.096813 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f0727-29f2-45fe-bbb0-de9d48c8f38a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:35.272294 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f0727-29f2-45fe-bbb0-de9d48c8f38a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:35.316270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:35.326517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78682590-00d1-4c6f-b4ed-6a504d06f86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:35.327190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78682590-00d1-4c6f-b4ed-6a504d06f86e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:35.327514 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78682590-00d1-4c6f-b4ed-6a504d06f86e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:35.327797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78682590-00d1-4c6f-b4ed-6a504d06f86e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:35.328720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78682590-00d1-4c6f-b4ed-6a504d06f86e None admin] RBAC: Adding query filter params (role.id=1df33ad923a44a59b50dcacb6a75fefb, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=f2c5536eddd346d692822a2f3befb48d) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:35.329037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78682590-00d1-4c6f-b4ed-6a504d06f86e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:35.330614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78682590-00d1-4c6f-b4ed-6a504d06f86e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:35.337527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 603/1219] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:33:35 2022] GET /identity/v3/role_assignments?user.id=f2c5536eddd346d692822a2f3befb48d&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=1df33ad923a44a59b50dcacb6a75fefb => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:33:37.684815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd563cc3-4061-4074-8725-cba73a31aaef None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:37.685369 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd563cc3-4061-4074-8725-cba73a31aaef None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:37.685834 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd563cc3-4061-4074-8725-cba73a31aaef None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:37.688726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 617/1220] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:33:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:33:37.697496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40528e-6edd-4f26-8c9c-eb8f19e50027 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:37.698133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40528e-6edd-4f26-8c9c-eb8f19e50027 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:37.698401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40528e-6edd-4f26-8c9c-eb8f19e50027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:37.721081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-1e40528e-6edd-4f26-8c9c-eb8f19e50027 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:33:37.722793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1e40528e-6edd-4f26-8c9c-eb8f19e50027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:33:37.819112 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 604/1221] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:33:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:33:37.856267 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd563cc3-4061-4074-8725-cba73a31aaef None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:38.011942 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd563cc3-4061-4074-8725-cba73a31aaef None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:38.051719 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56d40f-407b-4b6d-8dcc-8c50267dad51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:38.052501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56d40f-407b-4b6d-8dcc-8c50267dad51 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:38.052781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56d40f-407b-4b6d-8dcc-8c50267dad51 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:38.053104 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56d40f-407b-4b6d-8dcc-8c50267dad51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:38.068693 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56d40f-407b-4b6d-8dcc-8c50267dad51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-3b56d40f-407b-4b6d-8dcc-8c50267dad51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:33:38.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:38.077057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:38.079422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:38.079422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:38.079422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:38.079422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:38.079422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:33:38.079422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:33:38.079422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:33:38.079422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:33:38.079422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:33:38.079422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:33:38.079422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:33:38.079422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:33:38.079422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 618/1222] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:33:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:38.083350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40528e-6edd-4f26-8c9c-eb8f19e50027 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:38.245092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40528e-6edd-4f26-8c9c-eb8f19e50027 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:38.275133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:38.291898 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45cf33-3e83-4910-9088-31a70adc8056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:38.292849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45cf33-3e83-4910-9088-31a70adc8056 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:38.293211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45cf33-3e83-4910-9088-31a70adc8056 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:38.293701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45cf33-3e83-4910-9088-31a70adc8056 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:38.294640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45cf33-3e83-4910-9088-31a70adc8056 None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:38.295067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45cf33-3e83-4910-9088-31a70adc8056 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:38.296994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45cf33-3e83-4910-9088-31a70adc8056 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:38.308905 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 605/1223] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:33:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:38.316002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56d40f-407b-4b6d-8dcc-8c50267dad51 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:38.383843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:38.461107 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56d40f-407b-4b6d-8dcc-8c50267dad51 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:38.507798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804a3c5-0808-4f59-91b1-de3d46e1df8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:38.508739 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804a3c5-0808-4f59-91b1-de3d46e1df8e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:38.509125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804a3c5-0808-4f59-91b1-de3d46e1df8e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:38.509446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804a3c5-0808-4f59-91b1-de3d46e1df8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:38.524005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804a3c5-0808-4f59-91b1-de3d46e1df8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:38.525702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804a3c5-0808-4f59-91b1-de3d46e1df8e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:38.552402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-0804a3c5-0808-4f59-91b1-de3d46e1df8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:38.552402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:38.552402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:38.552402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:38.552402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:38.552402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:38.552402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:38.552402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:38.552402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:38.552402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:38.552402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:38.552402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:38.552402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:33:38.552402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:33:38.552402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:38.554463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:38.556199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:33:38.556199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:33:38.556199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:38.556199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:33:38.556199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 619/1224] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:38.562109 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45cf33-3e83-4910-9088-31a70adc8056 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:38.705210 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45cf33-3e83-4910-9088-31a70adc8056 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:38.760835 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3adbc0-f794-48da-8199-af39dcc6fc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:38.761494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3adbc0-f794-48da-8199-af39dcc6fc1c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:38.761863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3adbc0-f794-48da-8199-af39dcc6fc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:38.762066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3adbc0-f794-48da-8199-af39dcc6fc1c None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:38.762802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3adbc0-f794-48da-8199-af39dcc6fc1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:38.763165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3adbc0-f794-48da-8199-af39dcc6fc1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:38.764755 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3adbc0-f794-48da-8199-af39dcc6fc1c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:38.782416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 606/1225] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:33:38.789059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804a3c5-0808-4f59-91b1-de3d46e1df8e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:38.915443 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:38.924439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804a3c5-0808-4f59-91b1-de3d46e1df8e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:38.976600 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88668fb-a4e7-4156-b2a7-2dd435487a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:38.977710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88668fb-a4e7-4156-b2a7-2dd435487a55 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:38.978159 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88668fb-a4e7-4156-b2a7-2dd435487a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:38.978647 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88668fb-a4e7-4156-b2a7-2dd435487a55 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:38.995216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88668fb-a4e7-4156-b2a7-2dd435487a55 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:38.997981 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88668fb-a4e7-4156-b2a7-2dd435487a55 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-f88668fb-a4e7-4156-b2a7-2dd435487a55 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:33:39.021349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:39.022588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:39.028991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:39.028991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:39.028991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:33:39.028991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:33:39.028991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:33:39.028991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:33:39.028991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:33:39.028991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:33:39.028991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 10:33:39.028991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:33:39.028991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 620/1226] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:39.030454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3adbc0-f794-48da-8199-af39dcc6fc1c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:39.106787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:39.176286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3adbc0-f794-48da-8199-af39dcc6fc1c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:39.220235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5ec47-2ab0-4f02-b1f1-b5c6a630c6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:39.221054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5ec47-2ab0-4f02-b1f1-b5c6a630c6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:39.221655 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5ec47-2ab0-4f02-b1f1-b5c6a630c6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:39.222168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5ec47-2ab0-4f02-b1f1-b5c6a630c6f5 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:39.223557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5ec47-2ab0-4f02-b1f1-b5c6a630c6f5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:39.224143 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5ec47-2ab0-4f02-b1f1-b5c6a630c6f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:39.226777 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5ec47-2ab0-4f02-b1f1-b5c6a630c6f5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:39.261540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 607/1227] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Aug 3 10:33:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:39.264663 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88668fb-a4e7-4156-b2a7-2dd435487a55 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:39.396686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88668fb-a4e7-4156-b2a7-2dd435487a55 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:39.439465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ab24c-bf1b-4bc2-8523-c12a6ef98539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:39.440496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ab24c-bf1b-4bc2-8523-c12a6ef98539 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:39.441049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ab24c-bf1b-4bc2-8523-c12a6ef98539 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:39.441673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ab24c-bf1b-4bc2-8523-c12a6ef98539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:39.457487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ab24c-bf1b-4bc2-8523-c12a6ef98539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:39.459751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ab24c-bf1b-4bc2-8523-c12a6ef98539 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:39.489660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-dc3ab24c-bf1b-4bc2-8523-c12a6ef98539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:39.489660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:39.489660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:39.489660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:39.489660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:39.489660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:39.489660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:39.489660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:39.489660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:39.489660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:39.489660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:39.489660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:39.489660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:33:39.489660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:33:39.489660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:39.490709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:39.491631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:33:39.491631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:33:39.491631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:39.491631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:33:39.491631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 621/1228] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:33:39.498267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5ec47-2ab0-4f02-b1f1-b5c6a630c6f5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:39.660375 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5ec47-2ab0-4f02-b1f1-b5c6a630c6f5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:39.664502 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:39.710243 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d5fde-db7d-46e4-916c-084461b8de05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:39.711508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d5fde-db7d-46e4-916c-084461b8de05 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:39.711508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d5fde-db7d-46e4-916c-084461b8de05 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:39.712034 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d5fde-db7d-46e4-916c-084461b8de05 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:39.712369 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d5fde-db7d-46e4-916c-084461b8de05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:39.712703 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d5fde-db7d-46e4-916c-084461b8de05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:39.714370 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d5fde-db7d-46e4-916c-084461b8de05 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:39.729009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 608/1229] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:39.736586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ab24c-bf1b-4bc2-8523-c12a6ef98539 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:39.751093 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:39.880521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ab24c-bf1b-4bc2-8523-c12a6ef98539 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:39.922486 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2a3c2-4e81-44b6-8da0-354d332a0f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:39.923558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2a3c2-4e81-44b6-8da0-354d332a0f73 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:39.923842 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2a3c2-4e81-44b6-8da0-354d332a0f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:39.924116 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2a3c2-4e81-44b6-8da0-354d332a0f73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:39.932905 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2a3c2-4e81-44b6-8da0-354d332a0f73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:39.934813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2a3c2-4e81-44b6-8da0-354d332a0f73 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:39.950445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-b3a2a3c2-4e81-44b6-8da0-354d332a0f73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:33:39.950445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:39.950445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:39.950445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:39.950445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:39.950445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:39.950445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:39.950445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:39.950445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:39.950445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:39.950445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:39.950445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:39.950445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:33:39.950445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:33:39.950445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:39.952833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:39.955665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:33:39.955665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:33:39.955665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:33:39.955665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:33:39.955665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:33:39.955665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:33:39.955665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:33:39.955665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:33:39.955665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 622/1230] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:33:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:39.959445 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d5fde-db7d-46e4-916c-084461b8de05 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:40.109185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d5fde-db7d-46e4-916c-084461b8de05 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:40.150603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c7705-9585-41b3-a1d7-dde5998ce0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:40.151512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c7705-9585-41b3-a1d7-dde5998ce0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:40.151945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c7705-9585-41b3-a1d7-dde5998ce0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:40.152258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c7705-9585-41b3-a1d7-dde5998ce0c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:40.153069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c7705-9585-41b3-a1d7-dde5998ce0c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:40.153606 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c7705-9585-41b3-a1d7-dde5998ce0c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:40.155527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c7705-9585-41b3-a1d7-dde5998ce0c8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:40.172085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 609/1231] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:33:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:40.182104 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2a3c2-4e81-44b6-8da0-354d332a0f73 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:40.268145 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:40.314352 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2a3c2-4e81-44b6-8da0-354d332a0f73 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:40.365848 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcf9f2-5ce5-47b2-9144-05757e0f3fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:40.366810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcf9f2-5ce5-47b2-9144-05757e0f3fbc None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:40.367271 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcf9f2-5ce5-47b2-9144-05757e0f3fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:40.367753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcf9f2-5ce5-47b2-9144-05757e0f3fbc None admin] PATH_INFO: `/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/f2c5536eddd346d692822a2f3befb48d/roles/1df33ad923a44a59b50dcacb6a75fefb` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:40.388578 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bcf9f2-5ce5-47b2-9144-05757e0f3fbc None admin] RBAC: Authorizing `identity:create_grant(project_id=4272b859979d49c8b8f0c17ec9eb81bc, user_id=f2c5536eddd346d692822a2f3befb48d, role_id=1df33ad923a44a59b50dcacb6a75fefb)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:40.390263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bcf9f2-5ce5-47b2-9144-05757e0f3fbc None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:40.407490 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 623/1232] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:33:40 2022] PUT /identity/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/f2c5536eddd346d692822a2f3befb48d/roles/1df33ad923a44a59b50dcacb6a75fefb => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:33:42.603867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc9afa-4f12-426d-9df5-4b196f1511f1 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:42.604455 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc9afa-4f12-426d-9df5-4b196f1511f1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:42.604455 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc9afa-4f12-426d-9df5-4b196f1511f1 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:42.605808 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 610/1233] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:33:42.611782 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a53740-1206-4b13-8069-2bccba966a54 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:42.612234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a53740-1206-4b13-8069-2bccba966a54 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:42.612365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a53740-1206-4b13-8069-2bccba966a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:42.633299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f1a53740-1206-4b13-8069-2bccba966a54 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:33:42.634589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f1a53740-1206-4b13-8069-2bccba966a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:33:42.718702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 624/1234] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:33:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:33:42.744491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc9afa-4f12-426d-9df5-4b196f1511f1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:42.772300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:42.856947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc9afa-4f12-426d-9df5-4b196f1511f1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:42.901118 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2866a5d-9a02-4a6d-a74b-c5d447e42629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:42.901941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2866a5d-9a02-4a6d-a74b-c5d447e42629 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:42.902267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2866a5d-9a02-4a6d-a74b-c5d447e42629 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:42.902663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2866a5d-9a02-4a6d-a74b-c5d447e42629 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:42.913067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2866a5d-9a02-4a6d-a74b-c5d447e42629 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-f2866a5d-9a02-4a6d-a74b-c5d447e42629 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:33:42.917941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:42.919018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:42.922601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:42.922601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:42.922601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:42.922601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:42.922601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:33:42.922601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:33:42.922601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:33:42.922601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:33:42.922601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:33:42.922601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:33:42.922601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 10:33:42.922601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:42.922601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 611/1235] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 3 10:33:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:42.925603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a53740-1206-4b13-8069-2bccba966a54 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:42.977746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:43.037854 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a53740-1206-4b13-8069-2bccba966a54 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:43.077702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1c3b7-67fd-4cf3-808d-3fca4435110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:43.078332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1c3b7-67fd-4cf3-808d-3fca4435110d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:43.078658 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1c3b7-67fd-4cf3-808d-3fca4435110d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:43.079451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1c3b7-67fd-4cf3-808d-3fca4435110d None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:43.080205 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1c3b7-67fd-4cf3-808d-3fca4435110d None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:43.080511 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1c3b7-67fd-4cf3-808d-3fca4435110d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:43.081906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1c3b7-67fd-4cf3-808d-3fca4435110d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:43.093491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 625/1236] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 3 10:33:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:43.099735 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2866a5d-9a02-4a6d-a74b-c5d447e42629 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:43.203024 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:43.223780 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2866a5d-9a02-4a6d-a74b-c5d447e42629 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:43.267218 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db033d0a-edb7-4889-8170-4c84de9e276a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:43.267886 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db033d0a-edb7-4889-8170-4c84de9e276a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:43.268169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db033d0a-edb7-4889-8170-4c84de9e276a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:43.268444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db033d0a-edb7-4889-8170-4c84de9e276a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:43.280294 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db033d0a-edb7-4889-8170-4c84de9e276a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:43.281874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db033d0a-edb7-4889-8170-4c84de9e276a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:43.297971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-db033d0a-edb7-4889-8170-4c84de9e276a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:43.297971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:43.297971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:43.297971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:43.297971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:43.297971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:43.297971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:43.297971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:43.297971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:43.297971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:43.297971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:43.297971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:43.297971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:33:43.297971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:33:43.297971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:43.299302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:43.300788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:33:43.300788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:33:43.300788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:43.300788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:43.300788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 612/1237] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:43.305346 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1c3b7-67fd-4cf3-808d-3fca4435110d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:43.412801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1c3b7-67fd-4cf3-808d-3fca4435110d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:43.421927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:43.450722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca14cc-cec6-4daa-bae8-ff7a585d3b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:43.451303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca14cc-cec6-4daa-bae8-ff7a585d3b2f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:43.451592 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca14cc-cec6-4daa-bae8-ff7a585d3b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:43.451872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca14cc-cec6-4daa-bae8-ff7a585d3b2f None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:43.452583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca14cc-cec6-4daa-bae8-ff7a585d3b2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:43.452890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca14cc-cec6-4daa-bae8-ff7a585d3b2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:43.454335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca14cc-cec6-4daa-bae8-ff7a585d3b2f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:43.467162 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 626/1238] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:33:43.473497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db033d0a-edb7-4889-8170-4c84de9e276a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:43.580400 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db033d0a-edb7-4889-8170-4c84de9e276a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:43.616091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3305cdd-be8f-47f7-8c04-8b8bb531f033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:43.616841 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3305cdd-be8f-47f7-8c04-8b8bb531f033 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:43.617070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3305cdd-be8f-47f7-8c04-8b8bb531f033 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:43.617318 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3305cdd-be8f-47f7-8c04-8b8bb531f033 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:43.620457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:43.626433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3305cdd-be8f-47f7-8c04-8b8bb531f033 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:43.628084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3305cdd-be8f-47f7-8c04-8b8bb531f033 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-d3305cdd-be8f-47f7-8c04-8b8bb531f033 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:33:43.639628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:43.640759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:43.641804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:43.641804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:43.641804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:33:43.641804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:33:43.641804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:33:43.641804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:33:43.641804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:33:43.641804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:33:43.641804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 10:33:43.641804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:43.641804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 613/1239] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:43.649688 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca14cc-cec6-4daa-bae8-ff7a585d3b2f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:43.757416 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca14cc-cec6-4daa-bae8-ff7a585d3b2f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:43.792102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd104587-5ca2-4c51-94d2-71ad9f177448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:43.792833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd104587-5ca2-4c51-94d2-71ad9f177448 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:43.793194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd104587-5ca2-4c51-94d2-71ad9f177448 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:43.793578 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd104587-5ca2-4c51-94d2-71ad9f177448 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:43.794387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd104587-5ca2-4c51-94d2-71ad9f177448 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:43.794698 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd104587-5ca2-4c51-94d2-71ad9f177448 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:43.796118 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd104587-5ca2-4c51-94d2-71ad9f177448 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:43.814971 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 627/1240] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Aug 3 10:33:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:43.821400 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3305cdd-be8f-47f7-8c04-8b8bb531f033 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:43.929244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3305cdd-be8f-47f7-8c04-8b8bb531f033 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:43.970607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d0e0f9-3f4b-4e03-925f-82c5404807ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:43.971308 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0e0f9-3f4b-4e03-925f-82c5404807ae None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:43.971606 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0e0f9-3f4b-4e03-925f-82c5404807ae None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:43.972012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0e0f9-3f4b-4e03-925f-82c5404807ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:44.044793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:44.052144 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d0e0f9-3f4b-4e03-925f-82c5404807ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:44.054093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d0e0f9-3f4b-4e03-925f-82c5404807ae None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:44.165288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 614/1241] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:33:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 10:33:44.171439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd104587-5ca2-4c51-94d2-71ad9f177448 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:44.206956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:44.288891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd104587-5ca2-4c51-94d2-71ad9f177448 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:44.326832 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f4b42-85f5-4f3c-98c4-057f2e8bf417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:44.327401 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f4b42-85f5-4f3c-98c4-057f2e8bf417 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:44.327682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f4b42-85f5-4f3c-98c4-057f2e8bf417 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:44.327954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f4b42-85f5-4f3c-98c4-057f2e8bf417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:44.339485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f4b42-85f5-4f3c-98c4-057f2e8bf417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:44.341046 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f4b42-85f5-4f3c-98c4-057f2e8bf417 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:44.356444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-dc3f4b42-85f5-4f3c-98c4-057f2e8bf417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:44.356444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:44.356444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:44.356444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:44.356444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:44.356444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:44.356444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:44.356444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:44.356444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:44.356444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:44.356444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:44.356444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:44.356444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:33:44.356444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:33:44.356444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:44.357614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:44.358827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:33:44.358827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:33:44.358827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:44.358827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:33:44.358827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 628/1242] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:44.363546 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d0e0f9-3f4b-4e03-925f-82c5404807ae None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:44.465164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:44.473253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d0e0f9-3f4b-4e03-925f-82c5404807ae None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:44.515459 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34670d75-bd25-4e27-8d43-e08a7a00b43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:44.516888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34670d75-bd25-4e27-8d43-e08a7a00b43a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:44.517299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34670d75-bd25-4e27-8d43-e08a7a00b43a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:44.517689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34670d75-bd25-4e27-8d43-e08a7a00b43a None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:44.518603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34670d75-bd25-4e27-8d43-e08a7a00b43a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:44.518983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34670d75-bd25-4e27-8d43-e08a7a00b43a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:44.520668 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34670d75-bd25-4e27-8d43-e08a7a00b43a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:44.534197 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 615/1243] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:44.541378 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f4b42-85f5-4f3c-98c4-057f2e8bf417 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:44.633788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:44.662427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f4b42-85f5-4f3c-98c4-057f2e8bf417 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:44.701059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218bff0-ef5b-418a-bb08-7fdcad5838e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:44.701843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218bff0-ef5b-418a-bb08-7fdcad5838e6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:44.702202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218bff0-ef5b-418a-bb08-7fdcad5838e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:44.702569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218bff0-ef5b-418a-bb08-7fdcad5838e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:44.710969 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218bff0-ef5b-418a-bb08-7fdcad5838e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:44.712427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218bff0-ef5b-418a-bb08-7fdcad5838e6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:44.728803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-d218bff0-ef5b-418a-bb08-7fdcad5838e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:33:44.728803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:44.728803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:44.728803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:44.728803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:44.728803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:44.728803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:44.728803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:44.728803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:44.728803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:44.728803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:44.728803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:44.728803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:33:44.728803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:33:44.728803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:44.729965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:44.736382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:33:44.736382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:33:44.736382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:33:44.736382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:33:44.736382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:33:44.736382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:33:44.736382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:33:44.736382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:33:44.736382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 629/1244] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:33:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:44.738074 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34670d75-bd25-4e27-8d43-e08a7a00b43a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:44.856627 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34670d75-bd25-4e27-8d43-e08a7a00b43a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:44.894956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d692c90-0f7a-44be-ac90-144d717d2d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:44.895620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d692c90-0f7a-44be-ac90-144d717d2d17 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:44.895906 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d692c90-0f7a-44be-ac90-144d717d2d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:44.896183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d692c90-0f7a-44be-ac90-144d717d2d17 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:44.896939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d692c90-0f7a-44be-ac90-144d717d2d17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:44.897248 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d692c90-0f7a-44be-ac90-144d717d2d17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:44.898720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d692c90-0f7a-44be-ac90-144d717d2d17 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:44.913133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 616/1245] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:33:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:44.919394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218bff0-ef5b-418a-bb08-7fdcad5838e6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:45.063351 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218bff0-ef5b-418a-bb08-7fdcad5838e6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:45.104502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa8e14-4fd8-469e-9958-603b7f607d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:45.105133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa8e14-4fd8-469e-9958-603b7f607d0d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:45.105435 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa8e14-4fd8-469e-9958-603b7f607d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:45.105729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa8e14-4fd8-469e-9958-603b7f607d0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:45.106533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa8e14-4fd8-469e-9958-603b7f607d0d None admin] RBAC: Adding query filter params (role.id=1df33ad923a44a59b50dcacb6a75fefb, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=f2c5536eddd346d692822a2f3befb48d) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:45.106849 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa8e14-4fd8-469e-9958-603b7f607d0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:45.108296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa8e14-4fd8-469e-9958-603b7f607d0d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:45.116001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 630/1246] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:33:44 2022] GET /identity/v3/role_assignments?user.id=f2c5536eddd346d692822a2f3befb48d&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=1df33ad923a44a59b50dcacb6a75fefb => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:33:47.395488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a869f3b-e40f-4214-8000-c17f596d32ee None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:47.396064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a869f3b-e40f-4214-8000-c17f596d32ee None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:47.396171 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a869f3b-e40f-4214-8000-c17f596d32ee None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:47.397710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 617/1247] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:33:47.403244 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad40c98-d688-441a-9158-2586c0fadf7a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:47.403743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad40c98-d688-441a-9158-2586c0fadf7a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:47.403865 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad40c98-d688-441a-9158-2586c0fadf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:47.410526 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:47.423515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0ad40c98-d688-441a-9158-2586c0fadf7a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:33:47.424843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0ad40c98-d688-441a-9158-2586c0fadf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:33:47.502704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 631/1248] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:33:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:33:47.528202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a869f3b-e40f-4214-8000-c17f596d32ee None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:47.546578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:47.645926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a869f3b-e40f-4214-8000-c17f596d32ee None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:47.689031 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042d9c4-e623-4d25-a428-c73dec4f3b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:47.689739 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042d9c4-e623-4d25-a428-c73dec4f3b92 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:47.690011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042d9c4-e623-4d25-a428-c73dec4f3b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:47.690291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042d9c4-e623-4d25-a428-c73dec4f3b92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:47.701792 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042d9c4-e623-4d25-a428-c73dec4f3b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-1042d9c4-e623-4d25-a428-c73dec4f3b92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:33:47.706776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:47.708053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:47.709145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:47.709145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:47.709145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:47.709145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:47.709145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:33:47.709145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:33:47.709145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:33:47.709145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:33:47.709145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:33:47.709145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:33:47.709145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:33:47.709145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:47.709145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 618/1249] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:33:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:47.716821 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad40c98-d688-441a-9158-2586c0fadf7a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:47.825909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad40c98-d688-441a-9158-2586c0fadf7a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:47.858974 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:47.865181 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1004fec-f706-46c2-ab82-379103366cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:47.865878 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1004fec-f706-46c2-ab82-379103366cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:47.866213 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1004fec-f706-46c2-ab82-379103366cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:47.866511 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1004fec-f706-46c2-ab82-379103366cd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:47.867372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1004fec-f706-46c2-ab82-379103366cd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:47.867763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1004fec-f706-46c2-ab82-379103366cd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:47.869312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1004fec-f706-46c2-ab82-379103366cd8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:47.878858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 632/1250] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:33:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:47.885097 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042d9c4-e623-4d25-a428-c73dec4f3b92 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:47.984182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:48.008711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042d9c4-e623-4d25-a428-c73dec4f3b92 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:48.049164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189b3cc-b867-4ec9-a02e-a8601be18bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:48.049912 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189b3cc-b867-4ec9-a02e-a8601be18bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:48.050194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189b3cc-b867-4ec9-a02e-a8601be18bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:48.050466 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189b3cc-b867-4ec9-a02e-a8601be18bc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:48.063641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189b3cc-b867-4ec9-a02e-a8601be18bc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:48.065095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189b3cc-b867-4ec9-a02e-a8601be18bc0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:48.083251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-b189b3cc-b867-4ec9-a02e-a8601be18bc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:48.083251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:48.083251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:48.083251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:48.083251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:48.083251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:48.083251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:48.083251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:48.083251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:48.083251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:48.083251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:48.083251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:48.083251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:33:48.083251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:33:48.083251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:48.084322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:48.085239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:33:48.085239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:33:48.085239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:48.085239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:48.085239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 619/1251] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:48.090595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1004fec-f706-46c2-ab82-379103366cd8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:48.202718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1004fec-f706-46c2-ab82-379103366cd8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:48.244427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea174da-1e13-4581-908f-9caeed0d5a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:48.245015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea174da-1e13-4581-908f-9caeed0d5a4c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:48.245347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea174da-1e13-4581-908f-9caeed0d5a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:48.245688 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea174da-1e13-4581-908f-9caeed0d5a4c None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:48.246471 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea174da-1e13-4581-908f-9caeed0d5a4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:48.246820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea174da-1e13-4581-908f-9caeed0d5a4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:48.248526 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea174da-1e13-4581-908f-9caeed0d5a4c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:48.268221 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 633/1252] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:48.274150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189b3cc-b867-4ec9-a02e-a8601be18bc0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:48.384281 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189b3cc-b867-4ec9-a02e-a8601be18bc0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:48.412349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:48.421526 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2e220-86c3-441b-af37-e20de929dea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:48.422183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2e220-86c3-441b-af37-e20de929dea0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:48.423004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2e220-86c3-441b-af37-e20de929dea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:48.423282 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2e220-86c3-441b-af37-e20de929dea0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:48.431063 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2e220-86c3-441b-af37-e20de929dea0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:48.432534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2e220-86c3-441b-af37-e20de929dea0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-daf2e220-86c3-441b-af37-e20de929dea0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:33:48.445365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:48.447003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:48.448234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:48.448234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:48.448234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:33:48.448234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:33:48.448234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:33:48.448234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:33:48.448234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:33:48.448234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:33:48.448234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 10:33:48.448234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:48.448234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 620/1253] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:33:48.452441 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea174da-1e13-4581-908f-9caeed0d5a4c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:48.514286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:48.570734 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea174da-1e13-4581-908f-9caeed0d5a4c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:48.617189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7a5f9-ef6b-46dc-8e1a-d64510834914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:48.617980 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7a5f9-ef6b-46dc-8e1a-d64510834914 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:48.618347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7a5f9-ef6b-46dc-8e1a-d64510834914 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:48.618722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7a5f9-ef6b-46dc-8e1a-d64510834914 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:48.619683 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7a5f9-ef6b-46dc-8e1a-d64510834914 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:48.620069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7a5f9-ef6b-46dc-8e1a-d64510834914 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:48.621504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7a5f9-ef6b-46dc-8e1a-d64510834914 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:48.644953 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 634/1254] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Aug 3 10:33:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:33:48.650719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2e220-86c3-441b-af37-e20de929dea0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:48.770422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2e220-86c3-441b-af37-e20de929dea0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:48.811330 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f40633-2923-4cb6-8d22-ed9420ab627d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:48.812125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f40633-2923-4cb6-8d22-ed9420ab627d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:48.812509 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f40633-2923-4cb6-8d22-ed9420ab627d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:48.812920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f40633-2923-4cb6-8d22-ed9420ab627d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:48.878423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:48.894488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f40633-2923-4cb6-8d22-ed9420ab627d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:48.896095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f40633-2923-4cb6-8d22-ed9420ab627d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:49.020881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 621/1255] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:33:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 10:33:49.027070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7a5f9-ef6b-46dc-8e1a-d64510834914 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:49.160386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7a5f9-ef6b-46dc-8e1a-d64510834914 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:49.180403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:49.209639 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e0302-8542-4b9a-a54e-70c0a78d3d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:49.210494 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e0302-8542-4b9a-a54e-70c0a78d3d21 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:49.210815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e0302-8542-4b9a-a54e-70c0a78d3d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:49.211194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e0302-8542-4b9a-a54e-70c0a78d3d21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:49.229479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e0302-8542-4b9a-a54e-70c0a78d3d21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:49.232479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e0302-8542-4b9a-a54e-70c0a78d3d21 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:49.253715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ee7e0302-8542-4b9a-a54e-70c0a78d3d21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:49.253715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:49.253715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:49.253715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:49.253715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:49.253715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:49.253715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:49.253715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:49.253715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:49.253715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:49.253715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:49.253715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:49.253715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:33:49.253715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:33:49.253715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:49.254988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:49.256077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:33:49.256077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:33:49.256077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:49.256077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:33:49.256077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 635/1256] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:49.261654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f40633-2923-4cb6-8d22-ed9420ab627d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:49.367341 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:49.381093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f40633-2923-4cb6-8d22-ed9420ab627d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:49.418697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab871d4-4329-459b-992c-b1ad2d044427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:49.419441 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab871d4-4329-459b-992c-b1ad2d044427 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:49.419666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab871d4-4329-459b-992c-b1ad2d044427 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:49.420067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab871d4-4329-459b-992c-b1ad2d044427 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:49.420683 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab871d4-4329-459b-992c-b1ad2d044427 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:49.421031 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab871d4-4329-459b-992c-b1ad2d044427 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:49.423050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab871d4-4329-459b-992c-b1ad2d044427 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:49.435438 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 622/1257] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:49.441519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e0302-8542-4b9a-a54e-70c0a78d3d21 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:49.568674 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e0302-8542-4b9a-a54e-70c0a78d3d21 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:49.610500 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31fff7-46ec-423b-b5f4-82702b380f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:49.611163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31fff7-46ec-423b-b5f4-82702b380f84 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:49.611531 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31fff7-46ec-423b-b5f4-82702b380f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:49.611803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31fff7-46ec-423b-b5f4-82702b380f84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:49.619532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31fff7-46ec-423b-b5f4-82702b380f84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:49.621143 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31fff7-46ec-423b-b5f4-82702b380f84 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:49.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ac31fff7-46ec-423b-b5f4-82702b380f84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:33:49.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:49.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:49.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:49.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:49.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:49.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:49.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:49.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:49.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:49.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:49.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:49.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:33:49.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:33:49.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:49.636105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:49.637172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:33:49.637172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:33:49.637172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:33:49.637172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:33:49.637172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:33:49.637172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:33:49.637172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:33:49.637172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:33:49.637172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 636/1258] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:33:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:49.642628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab871d4-4329-459b-992c-b1ad2d044427 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:49.749342 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab871d4-4329-459b-992c-b1ad2d044427 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:49.784824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08dde3-0c61-494b-836b-7b6bec86da2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:49.785456 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08dde3-0c61-494b-836b-7b6bec86da2c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:49.785766 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08dde3-0c61-494b-836b-7b6bec86da2c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:49.786045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08dde3-0c61-494b-836b-7b6bec86da2c None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:49.786803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08dde3-0c61-494b-836b-7b6bec86da2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:49.787112 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08dde3-0c61-494b-836b-7b6bec86da2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:49.788537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08dde3-0c61-494b-836b-7b6bec86da2c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:49.802303 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 623/1259] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:33:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:49.808555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31fff7-46ec-423b-b5f4-82702b380f84 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:49.835441 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:49.917856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31fff7-46ec-423b-b5f4-82702b380f84 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:49.973039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6622b4-4f87-4fe9-be17-810e49f72808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:49.973728 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6622b4-4f87-4fe9-be17-810e49f72808 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:49.974064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6622b4-4f87-4fe9-be17-810e49f72808 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:49.974295 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6622b4-4f87-4fe9-be17-810e49f72808 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:49.975191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6622b4-4f87-4fe9-be17-810e49f72808 None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=f2c5536eddd346d692822a2f3befb48d) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:49.975545 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6622b4-4f87-4fe9-be17-810e49f72808 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:49.976923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6622b4-4f87-4fe9-be17-810e49f72808 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:49.983953 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 637/1260] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:33:49 2022] GET /identity/v3/role_assignments?user.id=f2c5536eddd346d692822a2f3befb48d&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=01714797cab143e1a7a150c068fa9315 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:52.118580 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852868f-c27c-4be5-b17e-cb72d74f562d None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:52.119129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852868f-c27c-4be5-b17e-cb72d74f562d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:52.119129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852868f-c27c-4be5-b17e-cb72d74f562d None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:52.122470 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 624/1261] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:33:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:33:52.128263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95d233-3527-400b-9a74-e65e369357d3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:52.128723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95d233-3527-400b-9a74-e65e369357d3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:52.128812 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95d233-3527-400b-9a74-e65e369357d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:52.148814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0a95d233-3527-400b-9a74-e65e369357d3 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:33:52.150080 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0a95d233-3527-400b-9a74-e65e369357d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:33:52.229674 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 638/1262] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:33:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:33:52.262439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852868f-c27c-4be5-b17e-cb72d74f562d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:52.276380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:52.372332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852868f-c27c-4be5-b17e-cb72d74f562d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:52.405896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab514f0e-e1a4-4947-b234-9b68ed80019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:52.406534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab514f0e-e1a4-4947-b234-9b68ed80019e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:52.407385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab514f0e-e1a4-4947-b234-9b68ed80019e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:52.407656 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab514f0e-e1a4-4947-b234-9b68ed80019e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:52.417817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab514f0e-e1a4-4947-b234-9b68ed80019e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-ab514f0e-e1a4-4947-b234-9b68ed80019e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:33:52.422761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:52.423863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:52.424783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:52.424783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:52.424783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:52.424783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:52.424783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:33:52.424783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:33:52.424783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:33:52.424783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:33:52.424783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:33:52.424783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:33:52.424783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:33:52.424783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:52.424783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 625/1263] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:33:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 10:33:52.430084 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95d233-3527-400b-9a74-e65e369357d3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:52.449348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:52.529787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95d233-3527-400b-9a74-e65e369357d3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:52.564225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bdf10e-ede9-4e08-ae2f-02f9e6f41fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:52.564846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bdf10e-ede9-4e08-ae2f-02f9e6f41fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:52.565127 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bdf10e-ede9-4e08-ae2f-02f9e6f41fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:52.565399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bdf10e-ede9-4e08-ae2f-02f9e6f41fd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:52.566151 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bdf10e-ede9-4e08-ae2f-02f9e6f41fd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:52.566454 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bdf10e-ede9-4e08-ae2f-02f9e6f41fd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:52.567816 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bdf10e-ede9-4e08-ae2f-02f9e6f41fd8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:52.577050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 639/1264] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:33:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:52.583423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab514f0e-e1a4-4947-b234-9b68ed80019e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:52.651467 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:52.687377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab514f0e-e1a4-4947-b234-9b68ed80019e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:52.722175 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60841d0b-fdb1-4713-b577-b29762db2c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:52.723232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60841d0b-fdb1-4713-b577-b29762db2c84 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:52.723690 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60841d0b-fdb1-4713-b577-b29762db2c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:52.723967 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60841d0b-fdb1-4713-b577-b29762db2c84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:52.737339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60841d0b-fdb1-4713-b577-b29762db2c84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:52.738836 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60841d0b-fdb1-4713-b577-b29762db2c84 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:52.754679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-60841d0b-fdb1-4713-b577-b29762db2c84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:52.754679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:52.754679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:52.754679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:52.754679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:52.754679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:52.754679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:52.754679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:52.754679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:52.754679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:52.754679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:52.754679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:52.754679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:33:52.754679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:33:52.754679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:52.755776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:52.756730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:33:52.756730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:33:52.756730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:52.756730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:52.756730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 626/1265] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:52.761724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bdf10e-ede9-4e08-ae2f-02f9e6f41fd8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:52.851391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:52.871733 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bdf10e-ede9-4e08-ae2f-02f9e6f41fd8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:52.904952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275f77f-4002-4ee4-91b4-2b977dd43968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:52.905513 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275f77f-4002-4ee4-91b4-2b977dd43968 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:52.905795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275f77f-4002-4ee4-91b4-2b977dd43968 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:52.906069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275f77f-4002-4ee4-91b4-2b977dd43968 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:52.906794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275f77f-4002-4ee4-91b4-2b977dd43968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:52.907093 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275f77f-4002-4ee4-91b4-2b977dd43968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:52.908560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275f77f-4002-4ee4-91b4-2b977dd43968 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:52.923452 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 640/1266] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:52.929944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60841d0b-fdb1-4713-b577-b29762db2c84 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:53.040855 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60841d0b-fdb1-4713-b577-b29762db2c84 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:53.061180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:53.075806 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c8803-234b-41f3-88a6-7f9d7082fffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:53.076454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c8803-234b-41f3-88a6-7f9d7082fffa None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:53.076734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c8803-234b-41f3-88a6-7f9d7082fffa None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:53.077001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c8803-234b-41f3-88a6-7f9d7082fffa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:53.084662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c8803-234b-41f3-88a6-7f9d7082fffa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:53.086129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c8803-234b-41f3-88a6-7f9d7082fffa None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-510c8803-234b-41f3-88a6-7f9d7082fffa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:33:53.097641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:53.098795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:53.099778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:53.099778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:53.099778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:33:53.099778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:33:53.099778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:33:53.099778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:33:53.099778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:33:53.099778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:33:53.099778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 10:33:53.099778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:53.099778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 627/1267] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:53.104810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275f77f-4002-4ee4-91b4-2b977dd43968 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:53.205993 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275f77f-4002-4ee4-91b4-2b977dd43968 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:53.244426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87626f-9917-4255-bcb5-e38383d55c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:53.245104 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87626f-9917-4255-bcb5-e38383d55c4e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:53.245386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87626f-9917-4255-bcb5-e38383d55c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:53.245820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87626f-9917-4255-bcb5-e38383d55c4e None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:53.246611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87626f-9917-4255-bcb5-e38383d55c4e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:53.246912 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87626f-9917-4255-bcb5-e38383d55c4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:53.248439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87626f-9917-4255-bcb5-e38383d55c4e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:53.258904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:53.271049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 641/1268] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Aug 3 10:33:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:33:53.277276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c8803-234b-41f3-88a6-7f9d7082fffa None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:53.383370 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c8803-234b-41f3-88a6-7f9d7082fffa None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:53.416665 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66f789-feeb-40e0-9695-8f6dbcdd03f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:53.417318 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66f789-feeb-40e0-9695-8f6dbcdd03f2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:53.417630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66f789-feeb-40e0-9695-8f6dbcdd03f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:53.417907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66f789-feeb-40e0-9695-8f6dbcdd03f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:53.432127 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66f789-feeb-40e0-9695-8f6dbcdd03f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:53.433603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66f789-feeb-40e0-9695-8f6dbcdd03f2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:53.448010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-8d66f789-feeb-40e0-9695-8f6dbcdd03f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:53.448010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:53.448010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:53.448010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:53.448010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:53.448010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:53.448010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:53.448010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:53.448010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:53.448010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:53.448010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:53.448010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:53.448010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:33:53.448010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:33:53.448010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:53.449169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:53.450498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:33:53.450498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:33:53.450498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:53.450498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:53.450498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 628/1269] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:53.455006 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87626f-9917-4255-bcb5-e38383d55c4e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:53.556638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87626f-9917-4255-bcb5-e38383d55c4e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:53.589478 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdcbc6-98f0-4971-acba-bd5cd60558ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:53.590125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdcbc6-98f0-4971-acba-bd5cd60558ce None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:53.590410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdcbc6-98f0-4971-acba-bd5cd60558ce None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:53.590686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdcbc6-98f0-4971-acba-bd5cd60558ce None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:53.591385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebdcbc6-98f0-4971-acba-bd5cd60558ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:53.591956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebdcbc6-98f0-4971-acba-bd5cd60558ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:53.593319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebdcbc6-98f0-4971-acba-bd5cd60558ce None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:53.604975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 642/1270] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:53.610702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66f789-feeb-40e0-9695-8f6dbcdd03f2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:53.641530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:53.708343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66f789-feeb-40e0-9695-8f6dbcdd03f2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:53.747587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638514a-e1aa-4cf1-bb47-b00bfab66927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:53.748233 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638514a-e1aa-4cf1-bb47-b00bfab66927 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:53.748514 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638514a-e1aa-4cf1-bb47-b00bfab66927 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:53.748791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638514a-e1aa-4cf1-bb47-b00bfab66927 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:53.755286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638514a-e1aa-4cf1-bb47-b00bfab66927 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:53.756724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638514a-e1aa-4cf1-bb47-b00bfab66927 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:53.768334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-e638514a-e1aa-4cf1-bb47-b00bfab66927 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:33:53.768334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:53.768334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:53.768334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:53.768334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:53.768334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:53.768334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:53.768334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:53.768334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:53.768334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:53.768334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:53.768334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:53.768334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:33:53.768334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:33:53.768334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:53.769411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:53.770565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:33:53.770565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:33:53.770565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:33:53.770565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:33:53.770565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:33:53.770565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:33:53.770565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:33:53.770565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:53.770565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 629/1271] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:33:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:53.775607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdcbc6-98f0-4971-acba-bd5cd60558ce None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:53.835470 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:53.877247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdcbc6-98f0-4971-acba-bd5cd60558ce None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:53.910510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129e8ce-686c-4da5-a66c-ae4c683f856a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:53.911147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129e8ce-686c-4da5-a66c-ae4c683f856a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:53.911429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129e8ce-686c-4da5-a66c-ae4c683f856a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:53.911705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129e8ce-686c-4da5-a66c-ae4c683f856a None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:53.912472 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129e8ce-686c-4da5-a66c-ae4c683f856a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:53.912780 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129e8ce-686c-4da5-a66c-ae4c683f856a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:53.914239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129e8ce-686c-4da5-a66c-ae4c683f856a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:53.927516 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 643/1272] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:33:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:53.938688 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638514a-e1aa-4cf1-bb47-b00bfab66927 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:54.041042 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:54.049927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638514a-e1aa-4cf1-bb47-b00bfab66927 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:54.082384 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38881c-27ca-4cba-a306-c2495e88bb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:54.083048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38881c-27ca-4cba-a306-c2495e88bb28 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:54.083331 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38881c-27ca-4cba-a306-c2495e88bb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:54.083607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38881c-27ca-4cba-a306-c2495e88bb28 None admin] PATH_INFO: `/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/f2c5536eddd346d692822a2f3befb48d/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:54.085572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38881c-27ca-4cba-a306-c2495e88bb28 None admin] RBAC: Authorizing `identity:create_grant(project_id=4272b859979d49c8b8f0c17ec9eb81bc, user_id=f2c5536eddd346d692822a2f3befb48d, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:54.086995 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38881c-27ca-4cba-a306-c2495e88bb28 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:54.099496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 630/1273] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 3 10:33:53 2022] PUT /identity/v3/projects/4272b859979d49c8b8f0c17ec9eb81bc/users/f2c5536eddd346d692822a2f3befb48d/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:33:56.100106 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a98fa-2167-40ce-b893-0229e20123c1 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:56.100667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a98fa-2167-40ce-b893-0229e20123c1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:56.100667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a98fa-2167-40ce-b893-0229e20123c1 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:56.102049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 644/1274] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:33:56.107266 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cae8d-4022-4123-a157-2a200e1e3b03 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:56.107716 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cae8d-4022-4123-a157-2a200e1e3b03 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:56.107818 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cae8d-4022-4123-a157-2a200e1e3b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:56.127161 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8e8cae8d-4022-4123-a157-2a200e1e3b03 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:33:56.128377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8e8cae8d-4022-4123-a157-2a200e1e3b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:33:56.196987 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 631/1275] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:33:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:33:56.221168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a98fa-2167-40ce-b893-0229e20123c1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:56.349454 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a98fa-2167-40ce-b893-0229e20123c1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:56.368351 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:56.385387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05320351-5fc5-4ecf-af16-9feacf9b8a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:56.386058 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05320351-5fc5-4ecf-af16-9feacf9b8a9a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:56.386340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05320351-5fc5-4ecf-af16-9feacf9b8a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:56.386613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05320351-5fc5-4ecf-af16-9feacf9b8a9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:56.396577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05320351-5fc5-4ecf-af16-9feacf9b8a9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-05320351-5fc5-4ecf-af16-9feacf9b8a9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 10:33:56.401405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:56.402523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:56.403451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:56.403451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:56.403451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:56.403451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:56.403451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 10:33:56.403451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 10:33:56.403451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 10:33:56.403451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 10:33:56.403451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 10:33:56.403451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 10:33:56.403451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 10:33:56.403451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:33:56.403451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 645/1276] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 3 10:33:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:56.409353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cae8d-4022-4123-a157-2a200e1e3b03 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:56.489972 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:56.508723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cae8d-4022-4123-a157-2a200e1e3b03 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:56.542269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e111c4-ba21-4feb-b814-cf049720f146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:56.542901 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e111c4-ba21-4feb-b814-cf049720f146 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:56.543186 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e111c4-ba21-4feb-b814-cf049720f146 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:56.543467 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e111c4-ba21-4feb-b814-cf049720f146 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:56.544184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e111c4-ba21-4feb-b814-cf049720f146 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:56.544490 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e111c4-ba21-4feb-b814-cf049720f146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:56.545887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e111c4-ba21-4feb-b814-cf049720f146 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:56.554305 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 632/1277] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 3 10:33:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:56.560049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05320351-5fc5-4ecf-af16-9feacf9b8a9a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:56.663163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05320351-5fc5-4ecf-af16-9feacf9b8a9a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:56.700495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff346b-c7c8-4cac-8bed-3ae4d50fc509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:56.701161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff346b-c7c8-4cac-8bed-3ae4d50fc509 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:56.701443 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff346b-c7c8-4cac-8bed-3ae4d50fc509 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:56.701755 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff346b-c7c8-4cac-8bed-3ae4d50fc509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:56.720166 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff346b-c7c8-4cac-8bed-3ae4d50fc509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:56.721668 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff346b-c7c8-4cac-8bed-3ae4d50fc509 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:56.737405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-21ff346b-c7c8-4cac-8bed-3ae4d50fc509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:56.737405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:56.737405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:56.737405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:56.737405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:56.737405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:56.737405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:56.737405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:56.737405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:56.737405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:56.737405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:56.737405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:56.737405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:33:56.737405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:33:56.737405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:56.738606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:56.739707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:33:56.739707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:33:56.739707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:56.739707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:33:56.739707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 646/1278] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:56.744461 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e111c4-ba21-4feb-b814-cf049720f146 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:56.844557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e111c4-ba21-4feb-b814-cf049720f146 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:56.880426 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c7469-ee91-4e6e-abd1-257c16f92fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:56.881053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c7469-ee91-4e6e-abd1-257c16f92fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:56.881333 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c7469-ee91-4e6e-abd1-257c16f92fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:56.881635 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c7469-ee91-4e6e-abd1-257c16f92fe3 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:56.882331 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c7469-ee91-4e6e-abd1-257c16f92fe3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:56.882635 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c7469-ee91-4e6e-abd1-257c16f92fe3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:56.884002 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c7469-ee91-4e6e-abd1-257c16f92fe3 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:56.895977 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 633/1279] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:56.901706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff346b-c7c8-4cac-8bed-3ae4d50fc509 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:56.932549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:57.002008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff346b-c7c8-4cac-8bed-3ae4d50fc509 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:57.035285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05ea1a-3c98-4147-8cd9-525640310312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:57.035938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05ea1a-3c98-4147-8cd9-525640310312 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:57.036221 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05ea1a-3c98-4147-8cd9-525640310312 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:57.036497 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05ea1a-3c98-4147-8cd9-525640310312 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:57.044989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05ea1a-3c98-4147-8cd9-525640310312 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:57.046485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05ea1a-3c98-4147-8cd9-525640310312 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-8f05ea1a-3c98-4147-8cd9-525640310312 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 10:33:57.059255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:57.060459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:57.061804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:57.061804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:57.061804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 10:33:57.061804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 10:33:57.061804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 10:33:57.061804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 10:33:57.061804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 10:33:57.061804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 10:33:57.061804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 10:33:57.061804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:33:57.061804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 647/1280] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:57.066858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c7469-ee91-4e6e-abd1-257c16f92fe3 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:57.071224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:57.171996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c7469-ee91-4e6e-abd1-257c16f92fe3 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:57.210974 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d85a55-4c5e-401d-9d31-2106eba3cbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:57.211605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d85a55-4c5e-401d-9d31-2106eba3cbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:57.211888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d85a55-4c5e-401d-9d31-2106eba3cbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:57.212164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d85a55-4c5e-401d-9d31-2106eba3cbe0 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:57.212980 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d85a55-4c5e-401d-9d31-2106eba3cbe0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:57.213288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d85a55-4c5e-401d-9d31-2106eba3cbe0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:57.214755 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d85a55-4c5e-401d-9d31-2106eba3cbe0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:57.240337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 634/1281] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Aug 3 10:33:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:57.246431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05ea1a-3c98-4147-8cd9-525640310312 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:57.339734 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:57.352282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05ea1a-3c98-4147-8cd9-525640310312 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:57.385930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d07bb-8414-4acd-bb47-d6f831e45e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:57.386575 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d07bb-8414-4acd-bb47-d6f831e45e62 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:57.386853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d07bb-8414-4acd-bb47-d6f831e45e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:57.387125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d07bb-8414-4acd-bb47-d6f831e45e62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:57.455298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d07bb-8414-4acd-bb47-d6f831e45e62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:57.456744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d07bb-8414-4acd-bb47-d6f831e45e62 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:57.546762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:57.562775 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 648/1282] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 3 10:33:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 10:33:57.569835 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d85a55-4c5e-401d-9d31-2106eba3cbe0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:57.636642 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:57.678520 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d85a55-4c5e-401d-9d31-2106eba3cbe0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:57.713931 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5049b70-ae58-422d-a0bf-a115303701e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:57.714588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5049b70-ae58-422d-a0bf-a115303701e2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:57.714866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5049b70-ae58-422d-a0bf-a115303701e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:57.715136 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5049b70-ae58-422d-a0bf-a115303701e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:57.726336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5049b70-ae58-422d-a0bf-a115303701e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:57.727807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5049b70-ae58-422d-a0bf-a115303701e2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:57.742722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-b5049b70-ae58-422d-a0bf-a115303701e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:57.742722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:57.742722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:57.742722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:57.742722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:57.742722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:57.742722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:57.742722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:57.742722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:57.742722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:57.742722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:57.742722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:57.742722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 10:33:57.742722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 10:33:57.742722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:57.743785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:57.745011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 10:33:57.745011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 10:33:57.745011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 10:33:57.745011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:57.745011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 635/1283] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:33:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:57.752053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d07bb-8414-4acd-bb47-d6f831e45e62 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:57.853784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d07bb-8414-4acd-bb47-d6f831e45e62 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:57.889313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03172fd6-c511-49f0-98c9-9712c000f632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:57.889956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03172fd6-c511-49f0-98c9-9712c000f632 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:57.890235 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03172fd6-c511-49f0-98c9-9712c000f632 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:57.890507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03172fd6-c511-49f0-98c9-9712c000f632 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:57.891762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03172fd6-c511-49f0-98c9-9712c000f632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:57.892181 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03172fd6-c511-49f0-98c9-9712c000f632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:57.893571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03172fd6-c511-49f0-98c9-9712c000f632 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:57.904881 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 649/1284] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 3 10:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:57.910616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5049b70-ae58-422d-a0bf-a115303701e2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:58.008176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5049b70-ae58-422d-a0bf-a115303701e2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:58.022474 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:58.042418 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925284e-4d81-4f51-83e6-bf065ff6627b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:58.043067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925284e-4d81-4f51-83e6-bf065ff6627b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:58.043350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925284e-4d81-4f51-83e6-bf065ff6627b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:58.043627 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925284e-4d81-4f51-83e6-bf065ff6627b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:58.050639 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925284e-4d81-4f51-83e6-bf065ff6627b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:58.052078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925284e-4d81-4f51-83e6-bf065ff6627b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:58.064611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-d925284e-4d81-4f51-83e6-bf065ff6627b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:33:58.064611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:33:58.064611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:33:58.064611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:33:58.064611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:33:58.064611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:33:58.064611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:33:58.064611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:33:58.064611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:33:58.064611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:33:58.064611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:33:58.064611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:33:58.064611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 10:33:58.064611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 10:33:58.064611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:33:58.065669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:33:58.066636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 10:33:58.066636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 10:33:58.066636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 10:33:58.066636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 10:33:58.066636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 10:33:58.066636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 10:33:58.066636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 10:33:58.066636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Aug 03 10:33:58.066636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 636/1285] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 3 10:33:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:33:58.071747 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03172fd6-c511-49f0-98c9-9712c000f632 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:58.128942 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:33:58.183131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03172fd6-c511-49f0-98c9-9712c000f632 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:58.216439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256b5ce-d67a-45c6-b06e-53a1c095f42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:58.217540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256b5ce-d67a-45c6-b06e-53a1c095f42c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:58.217660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256b5ce-d67a-45c6-b06e-53a1c095f42c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:58.218006 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256b5ce-d67a-45c6-b06e-53a1c095f42c None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:58.218846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256b5ce-d67a-45c6-b06e-53a1c095f42c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:58.219229 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256b5ce-d67a-45c6-b06e-53a1c095f42c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:58.220889 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256b5ce-d67a-45c6-b06e-53a1c095f42c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:58.238946 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 650/1286] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 3 10:33:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:33:58.249595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925284e-4d81-4f51-83e6-bf065ff6627b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:33:58.350937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925284e-4d81-4f51-83e6-bf065ff6627b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:33:58.383615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da7f9a-1faf-4e97-9aa1-be7a5510d0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:33:58.384160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da7f9a-1faf-4e97-9aa1-be7a5510d0bd None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:33:58.384440 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da7f9a-1faf-4e97-9aa1-be7a5510d0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:33:58.384711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da7f9a-1faf-4e97-9aa1-be7a5510d0bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:33:58.385508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da7f9a-1faf-4e97-9aa1-be7a5510d0bd None admin] RBAC: Adding query filter params (role.id=01714797cab143e1a7a150c068fa9315, scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc, user.id=f2c5536eddd346d692822a2f3befb48d) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:33:58.385819 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da7f9a-1faf-4e97-9aa1-be7a5510d0bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:33:58.387227 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da7f9a-1faf-4e97-9aa1-be7a5510d0bd None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:33:58.392838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 637/1287] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 3 10:33:58 2022] GET /identity/v3/role_assignments?user.id=f2c5536eddd346d692822a2f3befb48d&scope.project.id=4272b859979d49c8b8f0c17ec9eb81bc&role.id=01714797cab143e1a7a150c068fa9315 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:34:00.420473 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be3bd2-1b24-4733-9d3d-f23af828f3cb None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:00.420999 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be3bd2-1b24-4733-9d3d-f23af828f3cb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:00.421091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be3bd2-1b24-4733-9d3d-f23af828f3cb None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:00.422423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 651/1288] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:34:00.427453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d91ff6-6577-44cf-9561-87d623eb3df1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:00.427863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d91ff6-6577-44cf-9561-87d623eb3df1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:00.428000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d91ff6-6577-44cf-9561-87d623eb3df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:00.445586 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-56d91ff6-6577-44cf-9561-87d623eb3df1 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:34:00.446761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-56d91ff6-6577-44cf-9561-87d623eb3df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:34:00.476018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:34:00.511806 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 638/1289] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:34:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:34:00.534688 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be3bd2-1b24-4733-9d3d-f23af828f3cb None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:34:00.631395 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be3bd2-1b24-4733-9d3d-f23af828f3cb None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:34:00.642317 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:34:00.665904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57af3f9-0c2c-4edf-9857-1f4178125860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:34:00.666538 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57af3f9-0c2c-4edf-9857-1f4178125860 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:00.666815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57af3f9-0c2c-4edf-9857-1f4178125860 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:00.667084 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57af3f9-0c2c-4edf-9857-1f4178125860 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:00.667702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57af3f9-0c2c-4edf-9857-1f4178125860 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:34:00.669055 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57af3f9-0c2c-4edf-9857-1f4178125860 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-f57af3f9-0c2c-4edf-9857-1f4178125860 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:34:00.677813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:34:00.679008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:34:00.680064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:34:00.680064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:34:00.680064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:34:00.680064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:34:00.680064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:34:00.680064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:34:00.680064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:34:00.680064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:34:00.680064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:34:00.680064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:34:00.680064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:34:00.680064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:34:00.680064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 10:34:00.680064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:34:00.680064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 652/1290] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 3 10:34:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:34:00.685022 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d91ff6-6577-44cf-9561-87d623eb3df1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:34:00.782448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d91ff6-6577-44cf-9561-87d623eb3df1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:34:00.815507 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c442f2c-3581-46e8-a9f6-ca806a18effe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:34:00.816134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c442f2c-3581-46e8-a9f6-ca806a18effe None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:00.816417 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c442f2c-3581-46e8-a9f6-ca806a18effe None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:00.816693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c442f2c-3581-46e8-a9f6-ca806a18effe None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:00.817374 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c442f2c-3581-46e8-a9f6-ca806a18effe None admin] RBAC: Adding query filter params (name=placement) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:34:00.817710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c442f2c-3581-46e8-a9f6-ca806a18effe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:34:00.819092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c442f2c-3581-46e8-a9f6-ca806a18effe None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:34:00.824224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 639/1291] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Aug 3 10:34:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:34:00.831319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57af3f9-0c2c-4edf-9857-1f4178125860 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:34:00.928581 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57af3f9-0c2c-4edf-9857-1f4178125860 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:34:00.961801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5d4f5-0de7-45c0-b979-32dc04d9cee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:34:00.962411 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5d4f5-0de7-45c0-b979-32dc04d9cee1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:00.962689 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5d4f5-0de7-45c0-b979-32dc04d9cee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:00.962960 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5d4f5-0de7-45c0-b979-32dc04d9cee1 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:00.963635 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5d4f5-0de7-45c0-b979-32dc04d9cee1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:34:00.963938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5d4f5-0de7-45c0-b979-32dc04d9cee1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:34:00.965287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5d4f5-0de7-45c0-b979-32dc04d9cee1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:34:00.970037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 653/1292] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Aug 3 10:34:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:34:02.960795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef5b9d-5400-4323-881c-60f4998104ff None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:02.961311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef5b9d-5400-4323-881c-60f4998104ff None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:02.961432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef5b9d-5400-4323-881c-60f4998104ff None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:02.962742 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 640/1293] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:34:02.967966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a9371-81b0-4d7f-88d9-9111ece550bb None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:02.968362 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a9371-81b0-4d7f-88d9-9111ece550bb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:02.968530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a9371-81b0-4d7f-88d9-9111ece550bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:02.986286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0d3a9371-81b0-4d7f-88d9-9111ece550bb None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:34:02.987483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0d3a9371-81b0-4d7f-88d9-9111ece550bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:34:03.017194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:34:03.055023 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 654/1294] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:34:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:34:03.077613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef5b9d-5400-4323-881c-60f4998104ff None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:34:03.110404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:34:03.180142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef5b9d-5400-4323-881c-60f4998104ff None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:34:03.217362 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393af363-bfb5-4e97-b316-7d6b1021d399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:34:03.217947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393af363-bfb5-4e97-b316-7d6b1021d399 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:03.218232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393af363-bfb5-4e97-b316-7d6b1021d399 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:03.218546 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393af363-bfb5-4e97-b316-7d6b1021d399 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:03.219482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393af363-bfb5-4e97-b316-7d6b1021d399 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:34:03.221205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393af363-bfb5-4e97-b316-7d6b1021d399 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:34:03.239402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 641/1295] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 3 10:34:03 2022] POST /identity/v3/services => generated 246 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:34:05.260573 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a39f6b-0568-4445-aa15-fd0f2bd6e96c None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:05.261173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a39f6b-0568-4445-aa15-fd0f2bd6e96c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:05.261173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a39f6b-0568-4445-aa15-fd0f2bd6e96c None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:05.262558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 655/1296] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:34:05.268716 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33762e-6bf8-4b9c-b894-8b6912afb027 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:05.269209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33762e-6bf8-4b9c-b894-8b6912afb027 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:05.269300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33762e-6bf8-4b9c-b894-8b6912afb027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:05.288342 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0f33762e-6bf8-4b9c-b894-8b6912afb027 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:34:05.289578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0f33762e-6bf8-4b9c-b894-8b6912afb027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:34:05.366961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 642/1297] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:34:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:34:05.389817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a39f6b-0568-4445-aa15-fd0f2bd6e96c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:34:05.491954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a39f6b-0568-4445-aa15-fd0f2bd6e96c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:34:05.526646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272fc90-345d-477d-b35a-043854e14661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:34:05.527204 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272fc90-345d-477d-b35a-043854e14661 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:05.527493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272fc90-345d-477d-b35a-043854e14661 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:05.527769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272fc90-345d-477d-b35a-043854e14661 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:05.528404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272fc90-345d-477d-b35a-043854e14661 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:34:05.529798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272fc90-345d-477d-b35a-043854e14661 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-6272fc90-345d-477d-b35a-043854e14661 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:34:05.538627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:34:05.539814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:34:05.540677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:34:05.540677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:34:05.540677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:34:05.540677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:34:05.540677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:34:05.540677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:34:05.540677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:34:05.540677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:34:05.540677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:34:05.540677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:34:05.540677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:34:05.540677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:34:05.540677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 10:34:05.540677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:34:05.540677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 656/1298] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 3 10:34:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:34:05.545967 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33762e-6bf8-4b9c-b894-8b6912afb027 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:34:05.564944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:34:05.647133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33762e-6bf8-4b9c-b894-8b6912afb027 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:34:05.680839 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c7910-be63-45c8-9f9c-ac2d3acfa634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:34:05.681494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c7910-be63-45c8-9f9c-ac2d3acfa634 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:05.681788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c7910-be63-45c8-9f9c-ac2d3acfa634 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:05.682070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c7910-be63-45c8-9f9c-ac2d3acfa634 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:05.682754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c7910-be63-45c8-9f9c-ac2d3acfa634 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:34:05.683061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c7910-be63-45c8-9f9c-ac2d3acfa634 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:34:05.684427 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c7910-be63-45c8-9f9c-ac2d3acfa634 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:34:05.690507 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 643/1299] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Aug 3 10:34:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:34:05.696387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272fc90-345d-477d-b35a-043854e14661 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:34:05.728344 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:34:05.797550 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272fc90-345d-477d-b35a-043854e14661 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:34:05.829861 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a954c7-b1cb-4f99-b6ac-a1a0c9eab116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:34:05.830476 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a954c7-b1cb-4f99-b6ac-a1a0c9eab116 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:05.830754 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a954c7-b1cb-4f99-b6ac-a1a0c9eab116 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:05.831025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a954c7-b1cb-4f99-b6ac-a1a0c9eab116 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:05.831770 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a954c7-b1cb-4f99-b6ac-a1a0c9eab116 None admin] RBAC: Adding query filter params (interface=public, service_id=de3f3a264ce64b97b40946d844aacaa8, region_id=RegionOne) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:34:05.832075 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a954c7-b1cb-4f99-b6ac-a1a0c9eab116 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:34:05.833422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a954c7-b1cb-4f99-b6ac-a1a0c9eab116 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:34:05.838257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 657/1300] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 3 10:34:05 2022] GET /identity/v3/endpoints?service_id=de3f3a264ce64b97b40946d844aacaa8&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:34:05.844159 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c7910-be63-45c8-9f9c-ac2d3acfa634 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:34:05.927018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:34:05.945811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c7910-be63-45c8-9f9c-ac2d3acfa634 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:34:05.979639 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134f6fc-a19c-4dcb-a5b2-232deb297a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:34:05.980262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134f6fc-a19c-4dcb-a5b2-232deb297a45 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:05.980546 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134f6fc-a19c-4dcb-a5b2-232deb297a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:05.980820 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134f6fc-a19c-4dcb-a5b2-232deb297a45 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:05.981497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134f6fc-a19c-4dcb-a5b2-232deb297a45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:34:05.982857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134f6fc-a19c-4dcb-a5b2-232deb297a45 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:34:05.989763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 644/1301] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 3 10:34:05 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:34:07.989578 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b707fa1-abc8-4614-bce5-2ddd3f471941 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:07.990108 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b707fa1-abc8-4614-bce5-2ddd3f471941 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:07.990222 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b707fa1-abc8-4614-bce5-2ddd3f471941 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:07.991508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 658/1302] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:34:07.996625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26a923-f403-4dd5-9b5d-50d4d425cb56 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:07.997052 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26a923-f403-4dd5-9b5d-50d4d425cb56 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:07.997160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26a923-f403-4dd5-9b5d-50d4d425cb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:08.015421 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-fe26a923-f403-4dd5-9b5d-50d4d425cb56 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:34:08.017126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fe26a923-f403-4dd5-9b5d-50d4d425cb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:34:08.082789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 645/1303] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:34:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 10:34:08.105661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b707fa1-abc8-4614-bce5-2ddd3f471941 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:34:08.214590 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:34:08.223008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b707fa1-abc8-4614-bce5-2ddd3f471941 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:34:08.266753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a057ab4-ba91-4274-9663-062fd1a268c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:34:08.267461 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a057ab4-ba91-4274-9663-062fd1a268c6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:08.267786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a057ab4-ba91-4274-9663-062fd1a268c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:08.268029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a057ab4-ba91-4274-9663-062fd1a268c6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:08.268861 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a057ab4-ba91-4274-9663-062fd1a268c6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:34:08.270677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a057ab4-ba91-4274-9663-062fd1a268c6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-7a057ab4-ba91-4274-9663-062fd1a268c6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 10:34:08.280487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 10:34:08.281769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 10:34:08.282638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 10:34:08.282638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Aug 03 10:34:08.282638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 10:34:08.282638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Aug 03 10:34:08.282638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 10:34:08.282638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 10:34:08.282638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 10:34:08.282638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Aug 03 10:34:08.282638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 10:34:08.282638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Aug 03 10:34:08.282638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 10:34:08.282638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 10:34:08.282638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 10:34:08.282638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Aug 03 10:34:08.282638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 659/1304] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 3 10:34:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 10:34:08.287890 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26a923-f403-4dd5-9b5d-50d4d425cb56 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:34:08.367830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:34:08.391587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26a923-f403-4dd5-9b5d-50d4d425cb56 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:34:08.425193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf51a8c-a363-4278-bcb4-80cdea59e14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:34:08.425855 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf51a8c-a363-4278-bcb4-80cdea59e14c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:08.426142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf51a8c-a363-4278-bcb4-80cdea59e14c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:08.426422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf51a8c-a363-4278-bcb4-80cdea59e14c None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:08.427108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf51a8c-a363-4278-bcb4-80cdea59e14c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:34:08.427423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf51a8c-a363-4278-bcb4-80cdea59e14c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:34:08.428780 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf51a8c-a363-4278-bcb4-80cdea59e14c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:34:08.433953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 646/1305] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Aug 3 10:34:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:34:08.440470 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a057ab4-ba91-4274-9663-062fd1a268c6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:34:08.552088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a057ab4-ba91-4274-9663-062fd1a268c6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:34:08.586890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f05cd-0028-4e11-a8ed-85878d95bc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:34:08.587506 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f05cd-0028-4e11-a8ed-85878d95bc5b None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:34:08.587783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f05cd-0028-4e11-a8ed-85878d95bc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:34:08.588055 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f05cd-0028-4e11-a8ed-85878d95bc5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:34:08.588765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f05cd-0028-4e11-a8ed-85878d95bc5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:34:08.590150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f05cd-0028-4e11-a8ed-85878d95bc5b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:34:08.610511 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 660/1306] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:34:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:34:18.431306 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=30 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:34:18.605619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:15.091890 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f1ba3-d434-4dcc-841c-1d7a84408c6b None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:15.092416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f1ba3-d434-4dcc-841c-1d7a84408c6b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:15.092416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f1ba3-d434-4dcc-841c-1d7a84408c6b None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:15.097951 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 647/1307] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:35:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:35:15.100399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11686546-ed08-4649-a6cc-2f4f3a184f16 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:15.100904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11686546-ed08-4649-a6cc-2f4f3a184f16 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:15.100987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11686546-ed08-4649-a6cc-2f4f3a184f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:15.254423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-11686546-ed08-4649-a6cc-2f4f3a184f16 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:35:15.255665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-11686546-ed08-4649-a6cc-2f4f3a184f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:15.428389 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 661/1308] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:35:15.507193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc476f22-5d51-4aa1-bc17-700dd6742b6e None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:15.507852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc476f22-5d51-4aa1-bc17-700dd6742b6e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:15.508378 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc476f22-5d51-4aa1-bc17-700dd6742b6e None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:15.510448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 648/1309] 10.222.0.47 () {54 vars in 965 bytes} [Wed Aug 3 10:35:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:35:15.518066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88c78c-667c-474e-8f39-1a47337347fc None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:15.518444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88c78c-667c-474e-8f39-1a47337347fc None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:15.518613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88c78c-667c-474e-8f39-1a47337347fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:15.668586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ac88c78c-667c-474e-8f39-1a47337347fc None None] MFA Rules not processed for user `1c211cc19a2045bd9383d31c7e81ceac`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:35:15.670743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ac88c78c-667c-474e-8f39-1a47337347fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:15.713302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:15.799332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 662/1310] 10.222.0.47 () {58 vars in 1062 bytes} [Wed Aug 3 10:35:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:35:15.808288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc476f22-5d51-4aa1-bc17-700dd6742b6e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:35:15.810093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bc476f22-5d51-4aa1-bc17-700dd6742b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:15.944087 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc476f22-5d51-4aa1-bc17-700dd6742b6e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:35:16.040211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77688572-c96e-4228-b245-befbb802c49e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:35:16.041801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77688572-c96e-4228-b245-befbb802c49e service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:16.042747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77688572-c96e-4228-b245-befbb802c49e service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:16.043315 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77688572-c96e-4228-b245-befbb802c49e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:16.045796 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-77688572-c96e-4228-b245-befbb802c49e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:16.158423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:16.257286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77688572-c96e-4228-b245-befbb802c49e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:35:16.259338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77688572-c96e-4228-b245-befbb802c49e service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:35:16.317531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 649/1311] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:35:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:35:18.714640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75743277-17c0-47da-adbb-7215aff03f30 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:18.715449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75743277-17c0-47da-adbb-7215aff03f30 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:18.715449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75743277-17c0-47da-adbb-7215aff03f30 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:18.716645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 663/1312] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:35:18.723221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b78f5-f85c-4e91-82eb-c3bc7c803024 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:18.723756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b78f5-f85c-4e91-82eb-c3bc7c803024 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:18.723915 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b78f5-f85c-4e91-82eb-c3bc7c803024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:18.815570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b69b78f5-f85c-4e91-82eb-c3bc7c803024 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:35:18.816832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b69b78f5-f85c-4e91-82eb-c3bc7c803024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:18.923382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 650/1313] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:35:18.977808 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a87607-a36f-417e-b594-d2c315b96474 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:18.978096 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a87607-a36f-417e-b594-d2c315b96474 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:18.978374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a87607-a36f-417e-b594-d2c315b96474 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:18.979723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 664/1314] 10.222.0.47 () {54 vars in 965 bytes} [Wed Aug 3 10:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:35:18.985919 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e71f5-7618-43f2-a7a7-a2ff89fae5ac None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:18.986211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e71f5-7618-43f2-a7a7-a2ff89fae5ac None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:18.986493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e71f5-7618-43f2-a7a7-a2ff89fae5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:18.998530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:19.078311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-050e71f5-7618-43f2-a7a7-a2ff89fae5ac None None] MFA Rules not processed for user `1c211cc19a2045bd9383d31c7e81ceac`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:35:19.079590 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-050e71f5-7618-43f2-a7a7-a2ff89fae5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:19.180812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 651/1315] 10.222.0.47 () {58 vars in 1062 bytes} [Wed Aug 3 10:35:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:35:19.189684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a87607-a36f-417e-b594-d2c315b96474 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:35:19.192487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-81a87607-a36f-417e-b594-d2c315b96474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:19.326548 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a87607-a36f-417e-b594-d2c315b96474 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:35:19.404289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:19.407398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d869b7d-7867-421a-bd66-b4b1fa6ca7f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:35:19.408343 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d869b7d-7867-421a-bd66-b4b1fa6ca7f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:19.409012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d869b7d-7867-421a-bd66-b4b1fa6ca7f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:19.409652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d869b7d-7867-421a-bd66-b4b1fa6ca7f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:19.411427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9d869b7d-7867-421a-bd66-b4b1fa6ca7f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:19.536334 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d869b7d-7867-421a-bd66-b4b1fa6ca7f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:35:19.537702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d869b7d-7867-421a-bd66-b4b1fa6ca7f9 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:35:19.557810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 665/1316] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:35:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:35:22.195369 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2a3ea-811d-43c7-a986-22c90903583b None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:22.195924 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2a3ea-811d-43c7-a986-22c90903583b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:22.196042 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2a3ea-811d-43c7-a986-22c90903583b None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:22.197875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 652/1317] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:35:22.203381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256454f2-0d5f-4380-a085-c8d2937b1d6e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:22.203904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256454f2-0d5f-4380-a085-c8d2937b1d6e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:22.203904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256454f2-0d5f-4380-a085-c8d2937b1d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:22.277718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-256454f2-0d5f-4380-a085-c8d2937b1d6e None None] MFA Rules not processed for user `3829732fccee407fa473d0c55a5aa1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:35:22.279068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-256454f2-0d5f-4380-a085-c8d2937b1d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:22.336309 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:22.402383 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 666/1318] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:35:22.446339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2a3ea-811d-43c7-a986-22c90903583b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:35:22.447936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-54d2a3ea-811d-43c7-a986-22c90903583b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:22.560326 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2a3ea-811d-43c7-a986-22c90903583b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:35:22.563389 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:22.642136 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9618d3f0-2d89-4e6d-a565-dc43c144a9af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:35:22.642775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618d3f0-2d89-4e6d-a565-dc43c144a9af service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:22.643058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618d3f0-2d89-4e6d-a565-dc43c144a9af service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:22.643363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618d3f0-2d89-4e6d-a565-dc43c144a9af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:22.645055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9618d3f0-2d89-4e6d-a565-dc43c144a9af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:22.779501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9618d3f0-2d89-4e6d-a565-dc43c144a9af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:35:22.781526 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9618d3f0-2d89-4e6d-a565-dc43c144a9af service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:35:22.802941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 653/1319] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:35:22 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:35:25.767076 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae473cb-ed22-46b5-8848-7f745083ffbd None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:25.767951 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae473cb-ed22-46b5-8848-7f745083ffbd None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:25.767951 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae473cb-ed22-46b5-8848-7f745083ffbd None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:25.769060 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 667/1320] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:35:25.774929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369c0a5-56c2-45f9-abc9-67920cdccd9d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:25.775473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369c0a5-56c2-45f9-abc9-67920cdccd9d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:25.775473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369c0a5-56c2-45f9-abc9-67920cdccd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:25.819574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:25.866848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b369c0a5-56c2-45f9-abc9-67920cdccd9d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:35:25.868937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b369c0a5-56c2-45f9-abc9-67920cdccd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:26.014129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 654/1321] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:35:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:35:26.054345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae473cb-ed22-46b5-8848-7f745083ffbd None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:35:26.056040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6ae473cb-ed22-46b5-8848-7f745083ffbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:26.171782 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae473cb-ed22-46b5-8848-7f745083ffbd None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:35:26.251348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469ef73-31b7-43c8-a1b6-9d3d4c4cfb7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:35:26.251969 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469ef73-31b7-43c8-a1b6-9d3d4c4cfb7d service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:26.252254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469ef73-31b7-43c8-a1b6-9d3d4c4cfb7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:26.252532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469ef73-31b7-43c8-a1b6-9d3d4c4cfb7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:26.254224 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2469ef73-31b7-43c8-a1b6-9d3d4c4cfb7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:26.261957 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:26.367073 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469ef73-31b7-43c8-a1b6-9d3d4c4cfb7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:35:26.368509 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469ef73-31b7-43c8-a1b6-9d3d4c4cfb7d service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:35:26.392487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 668/1322] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:35:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:35:29.211339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628012b-1756-4e02-8d2b-51bc5697e9fd None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:29.211904 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628012b-1756-4e02-8d2b-51bc5697e9fd None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:29.211904 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628012b-1756-4e02-8d2b-51bc5697e9fd None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:29.213585 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 655/1323] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:35:29.219188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c5b96-f828-41b4-8094-0254565b470a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:29.219731 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c5b96-f828-41b4-8094-0254565b470a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:29.219731 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c5b96-f828-41b4-8094-0254565b470a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:29.299135 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d07c5b96-f828-41b4-8094-0254565b470a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:35:29.300366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d07c5b96-f828-41b4-8094-0254565b470a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:29.363554 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:29.418208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 669/1324] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:35:29.456653 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628012b-1756-4e02-8d2b-51bc5697e9fd None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:35:29.458375 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6628012b-1756-4e02-8d2b-51bc5697e9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:29.582477 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628012b-1756-4e02-8d2b-51bc5697e9fd None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:35:29.597839 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:29.703530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cedffe-89f2-4ca0-9199-15cb839225d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:35:29.704692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cedffe-89f2-4ca0-9199-15cb839225d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:29.705525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cedffe-89f2-4ca0-9199-15cb839225d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:29.706161 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cedffe-89f2-4ca0-9199-15cb839225d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:29.709625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-09cedffe-89f2-4ca0-9199-15cb839225d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:29.875774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cedffe-89f2-4ca0-9199-15cb839225d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:35:29.877242 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cedffe-89f2-4ca0-9199-15cb839225d6 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:35:29.903996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 656/1325] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:35:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:35:32.114846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c178b4f-0c87-44b5-b879-1322d84c3231 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:32.115561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c178b4f-0c87-44b5-b879-1322d84c3231 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:32.115690 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c178b4f-0c87-44b5-b879-1322d84c3231 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:32.117111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 670/1326] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:35:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:35:32.123050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9d30a-ab5e-42c9-91bf-dff15f841769 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:32.123576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9d30a-ab5e-42c9-91bf-dff15f841769 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:32.123736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9d30a-ab5e-42c9-91bf-dff15f841769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:32.177082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:32.198896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e0b9d30a-ab5e-42c9-91bf-dff15f841769 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:35:32.200119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e0b9d30a-ab5e-42c9-91bf-dff15f841769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:32.311897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 657/1327] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:35:32.354743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c178b4f-0c87-44b5-b879-1322d84c3231 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:35:32.356410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6c178b4f-0c87-44b5-b879-1322d84c3231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:32.486127 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c178b4f-0c87-44b5-b879-1322d84c3231 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:35:32.567301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63653c6f-1551-4637-9121-e11042b6708d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:35:32.567924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63653c6f-1551-4637-9121-e11042b6708d service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:32.568205 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63653c6f-1551-4637-9121-e11042b6708d service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:32.568479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63653c6f-1551-4637-9121-e11042b6708d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:32.570170 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-63653c6f-1551-4637-9121-e11042b6708d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:32.589317 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:32.739852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63653c6f-1551-4637-9121-e11042b6708d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:35:32.742499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63653c6f-1551-4637-9121-e11042b6708d service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:35:32.777966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 671/1328] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:35:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:35:35.026128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcba040-5344-4659-b740-8fdebe722399 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:35.027428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcba040-5344-4659-b740-8fdebe722399 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:35.027428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcba040-5344-4659-b740-8fdebe722399 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:35.029502 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 658/1329] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:35:35.036283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d9041-350d-4a76-b3ca-33f4c3e0fc9a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:35.036878 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d9041-350d-4a76-b3ca-33f4c3e0fc9a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:35.036878 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d9041-350d-4a76-b3ca-33f4c3e0fc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:35.118812 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c69d9041-350d-4a76-b3ca-33f4c3e0fc9a None None] MFA Rules not processed for user `3829732fccee407fa473d0c55a5aa1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:35:35.120045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c69d9041-350d-4a76-b3ca-33f4c3e0fc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:35.211488 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:35.251643 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 672/1330] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:35:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:35:35.317144 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcba040-5344-4659-b740-8fdebe722399 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:35:35.318709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7fcba040-5344-4659-b740-8fdebe722399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:35.441660 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcba040-5344-4659-b740-8fdebe722399 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:35:35.464199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:35.523674 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3dddb-cdbb-4abb-884a-4c076839376a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:35:35.524298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3dddb-cdbb-4abb-884a-4c076839376a service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:35.524582 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3dddb-cdbb-4abb-884a-4c076839376a service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:35.524851 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3dddb-cdbb-4abb-884a-4c076839376a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:35.526563 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a5a3dddb-cdbb-4abb-884a-4c076839376a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:35.687937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3dddb-cdbb-4abb-884a-4c076839376a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:35:35.689645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3dddb-cdbb-4abb-884a-4c076839376a service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:35:35.712753 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 659/1331] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:35:35 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:35:38.067591 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1397b01-5847-4a6c-8ac3-bb95c7ff0da0 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:38.068164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1397b01-5847-4a6c-8ac3-bb95c7ff0da0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:38.068164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1397b01-5847-4a6c-8ac3-bb95c7ff0da0 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:38.069559 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 673/1332] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:35:38.075554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bac6d-369b-4e3d-a425-6ad128665757 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:38.076106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bac6d-369b-4e3d-a425-6ad128665757 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:38.076106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bac6d-369b-4e3d-a425-6ad128665757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:38.145835 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:38.149433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7c4bac6d-369b-4e3d-a425-6ad128665757 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:35:38.151109 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7c4bac6d-369b-4e3d-a425-6ad128665757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:38.256625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 660/1333] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:35:38.301888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1397b01-5847-4a6c-8ac3-bb95c7ff0da0 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:35:38.303548 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a1397b01-5847-4a6c-8ac3-bb95c7ff0da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:38.420318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1397b01-5847-4a6c-8ac3-bb95c7ff0da0 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:35:38.496732 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ea413-769a-4662-a3d7-9ce8698ae358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:35:38.497518 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ea413-769a-4662-a3d7-9ce8698ae358 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:38.497900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ea413-769a-4662-a3d7-9ce8698ae358 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:38.498372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ea413-769a-4662-a3d7-9ce8698ae358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:38.500934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-681ea413-769a-4662-a3d7-9ce8698ae358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:38.531273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:38.624672 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ea413-769a-4662-a3d7-9ce8698ae358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:35:38.626143 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ea413-769a-4662-a3d7-9ce8698ae358 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:35:38.644373 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 674/1334] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:35:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:35:40.742332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e313a-42cb-4525-a653-5025928ffcac None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:40.743217 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e313a-42cb-4525-a653-5025928ffcac None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:40.743217 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e313a-42cb-4525-a653-5025928ffcac None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:40.744245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 661/1335] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:35:40.749803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17251d4-af21-46a2-86d5-eb26f14fa79d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:40.750292 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17251d4-af21-46a2-86d5-eb26f14fa79d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:40.750390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17251d4-af21-46a2-86d5-eb26f14fa79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:40.818105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c17251d4-af21-46a2-86d5-eb26f14fa79d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:35:40.819357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c17251d4-af21-46a2-86d5-eb26f14fa79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:40.898847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:40.923653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 675/1336] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:35:40.963856 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e313a-42cb-4525-a653-5025928ffcac None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:35:40.965432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-445e313a-42cb-4525-a653-5025928ffcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:41.086163 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e313a-42cb-4525-a653-5025928ffcac None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:35:41.121270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:41.174956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0a474-8029-44e0-91c1-ac5952639a6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:35:41.175607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0a474-8029-44e0-91c1-ac5952639a6b service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:41.175886 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0a474-8029-44e0-91c1-ac5952639a6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:41.176166 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0a474-8029-44e0-91c1-ac5952639a6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:41.177916 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bfc0a474-8029-44e0-91c1-ac5952639a6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:41.294472 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0a474-8029-44e0-91c1-ac5952639a6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:35:41.295824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0a474-8029-44e0-91c1-ac5952639a6b service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:35:41.314232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 662/1337] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:35:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:35:43.819555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060beaf-4804-47d7-8ca4-bfd7d70faad9 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:43.820209 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060beaf-4804-47d7-8ca4-bfd7d70faad9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:43.820209 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060beaf-4804-47d7-8ca4-bfd7d70faad9 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:43.821533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 676/1338] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:35:43.827925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb38d8-2bf2-4244-a165-55aaaae6dcf2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:43.828540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb38d8-2bf2-4244-a165-55aaaae6dcf2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:43.828540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb38d8-2bf2-4244-a165-55aaaae6dcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:43.904108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-bddb38d8-2bf2-4244-a165-55aaaae6dcf2 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:35:43.905955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bddb38d8-2bf2-4244-a165-55aaaae6dcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:43.916613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:44.024778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 663/1339] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:35:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:35:44.062360 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060beaf-4804-47d7-8ca4-bfd7d70faad9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:35:44.064052 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7060beaf-4804-47d7-8ca4-bfd7d70faad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:44.179708 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060beaf-4804-47d7-8ca4-bfd7d70faad9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:35:44.260558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe895c-9816-4cd5-b0e2-c217c811d0d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:35:44.261190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe895c-9816-4cd5-b0e2-c217c811d0d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:44.261506 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe895c-9816-4cd5-b0e2-c217c811d0d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:44.261791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe895c-9816-4cd5-b0e2-c217c811d0d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:44.263481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f9fe895c-9816-4cd5-b0e2-c217c811d0d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:44.306063 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:44.383046 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe895c-9816-4cd5-b0e2-c217c811d0d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:35:44.384479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe895c-9816-4cd5-b0e2-c217c811d0d8 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:35:44.403272 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 677/1340] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:35:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:35:48.681452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f3ade-4723-49e6-af24-bb87d8224040 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:48.681990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f3ade-4723-49e6-af24-bb87d8224040 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:48.682071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f3ade-4723-49e6-af24-bb87d8224040 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:48.683440 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 664/1341] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:35:48.689653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf22053-bdb1-4abf-8720-739afa609abb None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:48.690118 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf22053-bdb1-4abf-8720-739afa609abb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:48.690212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf22053-bdb1-4abf-8720-739afa609abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:48.768592 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-fbf22053-bdb1-4abf-8720-739afa609abb None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:35:48.769776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fbf22053-bdb1-4abf-8720-739afa609abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:48.880320 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:48.894481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 678/1342] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:35:48.935417 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f3ade-4723-49e6-af24-bb87d8224040 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:35:48.939290 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-308f3ade-4723-49e6-af24-bb87d8224040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:49.056858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f3ade-4723-49e6-af24-bb87d8224040 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:35:49.107817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:49.148450 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af977d1-b2a1-4846-9039-5dca61e3c556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:35:49.149188 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af977d1-b2a1-4846-9039-5dca61e3c556 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:49.149572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af977d1-b2a1-4846-9039-5dca61e3c556 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:49.149850 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af977d1-b2a1-4846-9039-5dca61e3c556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:49.151575 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9af977d1-b2a1-4846-9039-5dca61e3c556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:49.288411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af977d1-b2a1-4846-9039-5dca61e3c556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:35:49.290057 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af977d1-b2a1-4846-9039-5dca61e3c556 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:35:49.309664 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 665/1343] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:35:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:35:52.588685 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4fa6d-912a-483d-a253-b2060085a61c None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:52.589276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4fa6d-912a-483d-a253-b2060085a61c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:52.589276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4fa6d-912a-483d-a253-b2060085a61c None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:52.590659 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 679/1344] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:35:52.598304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db1bb6-a6a6-41b3-bccb-5a59b811e6bb None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:52.598838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db1bb6-a6a6-41b3-bccb-5a59b811e6bb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:52.599116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db1bb6-a6a6-41b3-bccb-5a59b811e6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:52.696856 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-99db1bb6-a6a6-41b3-bccb-5a59b811e6bb None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:35:52.698124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-99db1bb6-a6a6-41b3-bccb-5a59b811e6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:52.719562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:52.827648 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 666/1345] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:35:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:35:52.867406 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4fa6d-912a-483d-a253-b2060085a61c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:35:52.869103 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ccf4fa6d-912a-483d-a253-b2060085a61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:52.988924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4fa6d-912a-483d-a253-b2060085a61c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:35:53.077379 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19566993-2372-447a-9ed4-7dd0716337bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:35:53.078054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19566993-2372-447a-9ed4-7dd0716337bb service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:53.078405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19566993-2372-447a-9ed4-7dd0716337bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:53.078405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19566993-2372-447a-9ed4-7dd0716337bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:53.080106 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-19566993-2372-447a-9ed4-7dd0716337bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:53.139711 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:53.221608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19566993-2372-447a-9ed4-7dd0716337bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:35:53.223103 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19566993-2372-447a-9ed4-7dd0716337bb service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:35:53.246827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 680/1346] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:35:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:35:57.832084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38746539-a51b-462b-b1cf-88f682d8155e None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:57.832827 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38746539-a51b-462b-b1cf-88f682d8155e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:57.833413 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38746539-a51b-462b-b1cf-88f682d8155e None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:57.836084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 667/1347] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:35:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:35:57.841867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af365c-f90d-406d-b399-df4e5453e82f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:57.842915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af365c-f90d-406d-b399-df4e5453e82f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:57.843016 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af365c-f90d-406d-b399-df4e5453e82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:57.924247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-78af365c-f90d-406d-b399-df4e5453e82f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:35:57.925503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-78af365c-f90d-406d-b399-df4e5453e82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:58.040976 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:58.044601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 681/1348] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:35:58.085519 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38746539-a51b-462b-b1cf-88f682d8155e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:35:58.087070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-38746539-a51b-462b-b1cf-88f682d8155e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:58.204487 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38746539-a51b-462b-b1cf-88f682d8155e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:35:58.261311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:35:58.301720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e16a8f-c646-41a6-b3d4-7f27bc86fb8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:35:58.302356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e16a8f-c646-41a6-b3d4-7f27bc86fb8c service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:35:58.302635 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e16a8f-c646-41a6-b3d4-7f27bc86fb8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:35:58.302905 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e16a8f-c646-41a6-b3d4-7f27bc86fb8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:35:58.304550 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-80e16a8f-c646-41a6-b3d4-7f27bc86fb8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:35:58.421520 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e16a8f-c646-41a6-b3d4-7f27bc86fb8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:35:58.422967 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e16a8f-c646-41a6-b3d4-7f27bc86fb8c service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:35:58.442377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 668/1349] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:36:03.854590 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17933fdb-dbbe-48cd-8347-dd149a86ce28 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:03.855318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17933fdb-dbbe-48cd-8347-dd149a86ce28 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:03.855318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17933fdb-dbbe-48cd-8347-dd149a86ce28 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:03.857609 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 682/1350] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 3 10:36:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:03.866246 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b66c0b-dfba-4d17-ac36-f6f5b48e860c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:03.867585 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b66c0b-dfba-4d17-ac36-f6f5b48e860c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:03.868312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b66c0b-dfba-4d17-ac36-f6f5b48e860c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:03.973111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-79b66c0b-dfba-4d17-ac36-f6f5b48e860c None None] MFA Rules not processed for user `f2c5536eddd346d692822a2f3befb48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:03.974678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-79b66c0b-dfba-4d17-ac36-f6f5b48e860c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:03.999423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:04.176537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 669/1351] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 3 10:36:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:08.927265 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f64ba9-803e-4a25-84ec-3be989e327fe None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:08.927794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f64ba9-803e-4a25-84ec-3be989e327fe None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:08.927794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f64ba9-803e-4a25-84ec-3be989e327fe None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:08.930165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 683/1352] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 3 10:36:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:08.938139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2795680-dc7f-4423-a215-d0891ca35630 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:08.939751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2795680-dc7f-4423-a215-d0891ca35630 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:08.939751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2795680-dc7f-4423-a215-d0891ca35630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:09.087669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e2795680-dc7f-4423-a215-d0891ca35630 None None] MFA Rules not processed for user `f2c5536eddd346d692822a2f3befb48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:09.088938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e2795680-dc7f-4423-a215-d0891ca35630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:09.189689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:09.246652 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 670/1353] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 3 10:36:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:10.918391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d62af-effd-446f-8edb-60360beb550d None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:10.918974 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d62af-effd-446f-8edb-60360beb550d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:10.918974 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d62af-effd-446f-8edb-60360beb550d None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:10.920314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 684/1354] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 3 10:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:36:10.926133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ef6b4-c65a-454d-9de4-7f9f6215eade None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:10.926701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ef6b4-c65a-454d-9de4-7f9f6215eade None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:10.926893 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ef6b4-c65a-454d-9de4-7f9f6215eade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:11.044819 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e53ef6b4-c65a-454d-9de4-7f9f6215eade None None] MFA Rules not processed for user `f2c5536eddd346d692822a2f3befb48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:11.046065 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e53ef6b4-c65a-454d-9de4-7f9f6215eade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:11.191333 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 671/1355] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 3 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:14.593427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a3841-6b5c-459e-baf6-813017bd1db5 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:14.594419 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a3841-6b5c-459e-baf6-813017bd1db5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:14.594419 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a3841-6b5c-459e-baf6-813017bd1db5 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:14.595715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 685/1356] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:14.601304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7d640-8bb0-4e1b-8b55-9e7de6a69834 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:14.601811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7d640-8bb0-4e1b-8b55-9e7de6a69834 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:14.601915 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7d640-8bb0-4e1b-8b55-9e7de6a69834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:14.648863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:14.691301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-70d7d640-8bb0-4e1b-8b55-9e7de6a69834 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:14.692582 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-70d7d640-8bb0-4e1b-8b55-9e7de6a69834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:14.835533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 672/1357] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:14.897987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b7dfa-5e43-4860-a399-de2771b435c1 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:14.898611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b7dfa-5e43-4860-a399-de2771b435c1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:14.899772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b7dfa-5e43-4860-a399-de2771b435c1 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:14.902290 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 686/1358] 10.222.0.47 () {54 vars in 962 bytes} [Wed Aug 3 10:36:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:14.913218 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8344771-2e57-47e5-9682-59f5b3ff5cc8 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:14.914071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8344771-2e57-47e5-9682-59f5b3ff5cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:14.914543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8344771-2e57-47e5-9682-59f5b3ff5cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:15.021634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e8344771-2e57-47e5-9682-59f5b3ff5cc8 None None] MFA Rules not processed for user `e66f25cc329f418eba6bd637bf3e8a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:15.022926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e8344771-2e57-47e5-9682-59f5b3ff5cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:15.074648 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:15.153788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 673/1359] 10.222.0.47 () {58 vars in 1059 bytes} [Wed Aug 3 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:15.162028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b7dfa-5e43-4860-a399-de2771b435c1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:36:15.163773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fe4b7dfa-5e43-4860-a399-de2771b435c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:15.329193 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b7dfa-5e43-4860-a399-de2771b435c1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:36:15.411612 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:15.456174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57d116-384b-4ad2-bb83-320bf85da752 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:36:15.456884 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57d116-384b-4ad2-bb83-320bf85da752 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:15.457232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57d116-384b-4ad2-bb83-320bf85da752 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:15.457614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57d116-384b-4ad2-bb83-320bf85da752 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:15.459794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fe57d116-384b-4ad2-bb83-320bf85da752 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:15.561032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57d116-384b-4ad2-bb83-320bf85da752 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:36:15.562762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57d116-384b-4ad2-bb83-320bf85da752 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:36:15.583859 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 687/1360] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:36:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:36:17.974380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a7046-d701-43a0-bce2-ebf257cef895 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:17.974838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a7046-d701-43a0-bce2-ebf257cef895 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:17.975173 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a7046-d701-43a0-bce2-ebf257cef895 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:17.976476 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 674/1361] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:17.981826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b66814-2135-434b-8d58-8301422bbf3e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:17.982305 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b66814-2135-434b-8d58-8301422bbf3e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:17.982419 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b66814-2135-434b-8d58-8301422bbf3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:18.067898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-50b66814-2135-434b-8d58-8301422bbf3e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:18.069282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-50b66814-2135-434b-8d58-8301422bbf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:18.114110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:18.202374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 688/1362] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:18.250544 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a7046-d701-43a0-bce2-ebf257cef895 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:36:18.252265 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-223a7046-d701-43a0-bce2-ebf257cef895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:18.463876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a7046-d701-43a0-bce2-ebf257cef895 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:36:18.475646 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:18.618284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71081f62-5e59-4d9e-9821-598ce50b270f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:36:18.619254 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71081f62-5e59-4d9e-9821-598ce50b270f service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:18.619652 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71081f62-5e59-4d9e-9821-598ce50b270f service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:18.620040 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71081f62-5e59-4d9e-9821-598ce50b270f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:18.621875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-71081f62-5e59-4d9e-9821-598ce50b270f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:18.755434 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71081f62-5e59-4d9e-9821-598ce50b270f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:36:18.758253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71081f62-5e59-4d9e-9821-598ce50b270f service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:36:18.791258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 675/1363] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:36:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:36:21.171666 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4fa50-9885-46aa-84a8-810eb14a54dd None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:21.172232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4fa50-9885-46aa-84a8-810eb14a54dd None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:21.172232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4fa50-9885-46aa-84a8-810eb14a54dd None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:21.173946 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 689/1364] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:21.179489 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e78d0e-cf46-4eee-a8ab-8f2cc72e2008 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:21.179996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e78d0e-cf46-4eee-a8ab-8f2cc72e2008 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:21.180089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e78d0e-cf46-4eee-a8ab-8f2cc72e2008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:21.197277 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:21.260523 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-12e78d0e-cf46-4eee-a8ab-8f2cc72e2008 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:21.261789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-12e78d0e-cf46-4eee-a8ab-8f2cc72e2008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:21.369146 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 676/1365] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:21.414520 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64ead7-0fde-479b-9695-057713c55fa6 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:21.415154 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64ead7-0fde-479b-9695-057713c55fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:21.415429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64ead7-0fde-479b-9695-057713c55fa6 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:21.416825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 690/1366] 10.222.0.47 () {54 vars in 962 bytes} [Wed Aug 3 10:36:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:21.423378 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636f718-df45-4ac8-9428-683d19c84999 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:21.423612 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636f718-df45-4ac8-9428-683d19c84999 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:21.423896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636f718-df45-4ac8-9428-683d19c84999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:21.513810 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3636f718-df45-4ac8-9428-683d19c84999 None None] MFA Rules not processed for user `e66f25cc329f418eba6bd637bf3e8a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:21.515035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3636f718-df45-4ac8-9428-683d19c84999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:21.553001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:21.643052 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 677/1367] 10.222.0.47 () {58 vars in 1059 bytes} [Wed Aug 3 10:36:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:21.651114 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64ead7-0fde-479b-9695-057713c55fa6 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:36:21.652963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9e64ead7-0fde-479b-9695-057713c55fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:21.837208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64ead7-0fde-479b-9695-057713c55fa6 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:36:21.869831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:21.991156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354588c-87a2-41d3-945d-5427eecd7dd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:36:21.991800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354588c-87a2-41d3-945d-5427eecd7dd4 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:21.992082 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354588c-87a2-41d3-945d-5427eecd7dd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:21.992357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354588c-87a2-41d3-945d-5427eecd7dd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:21.994704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1354588c-87a2-41d3-945d-5427eecd7dd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:22.104880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354588c-87a2-41d3-945d-5427eecd7dd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:36:22.106586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354588c-87a2-41d3-945d-5427eecd7dd4 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:36:22.129513 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 691/1368] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:36:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:36:23.070537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5a64e-3c9d-49d9-98be-695fcfa3bcd9 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:23.071308 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5a64e-3c9d-49d9-98be-695fcfa3bcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:23.071308 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5a64e-3c9d-49d9-98be-695fcfa3bcd9 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:23.072661 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 678/1369] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:23.078659 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbf49f-7564-4299-bd68-976ffe5bcdcc None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:23.079209 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbf49f-7564-4299-bd68-976ffe5bcdcc None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:23.079209 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbf49f-7564-4299-bd68-976ffe5bcdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:23.115143 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:23.153540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-39fbf49f-7564-4299-bd68-976ffe5bcdcc None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:23.154831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-39fbf49f-7564-4299-bd68-976ffe5bcdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:23.269849 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 692/1370] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:23.615410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4edff3-298c-4e54-b565-1054a6804f63 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:23.615701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4edff3-298c-4e54-b565-1054a6804f63 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:23.615977 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4edff3-298c-4e54-b565-1054a6804f63 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:23.617634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 679/1371] 10.222.0.47 () {54 vars in 964 bytes} [Wed Aug 3 10:36:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:23.622631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718f26d-5a59-4a65-90ad-3aa8483e7052 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:23.623223 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718f26d-5a59-4a65-90ad-3aa8483e7052 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:23.623713 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718f26d-5a59-4a65-90ad-3aa8483e7052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:23.725628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0718f26d-5a59-4a65-90ad-3aa8483e7052 None None] MFA Rules not processed for user `3b5f547a388c4f64aeb95a7435818741`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:23.726924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0718f26d-5a59-4a65-90ad-3aa8483e7052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:23.774951 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:23.806326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 693/1372] 10.222.0.47 () {58 vars in 1061 bytes} [Wed Aug 3 10:36:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:23.814015 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4edff3-298c-4e54-b565-1054a6804f63 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:36:23.815539 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0c4edff3-298c-4e54-b565-1054a6804f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:23.928311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4edff3-298c-4e54-b565-1054a6804f63 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:36:23.970407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:24.008234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09416f-41e3-47ea-a451-01ce4f8c957c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:36:24.008867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09416f-41e3-47ea-a451-01ce4f8c957c service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:24.009148 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09416f-41e3-47ea-a451-01ce4f8c957c service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:24.009421 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09416f-41e3-47ea-a451-01ce4f8c957c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:24.011083 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9d09416f-41e3-47ea-a451-01ce4f8c957c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:24.131327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d09416f-41e3-47ea-a451-01ce4f8c957c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:36:24.132590 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d09416f-41e3-47ea-a451-01ce4f8c957c service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:36:24.152756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 680/1373] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:36:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:36:24.647424 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840f517-97fc-47b4-bae2-f629cbddd79e None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:24.647926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840f517-97fc-47b4-bae2-f629cbddd79e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:24.647926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840f517-97fc-47b4-bae2-f629cbddd79e None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:24.649281 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 694/1374] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:24.654565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c05c4-2a54-44eb-97ac-f6e031c99012 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:24.654880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c05c4-2a54-44eb-97ac-f6e031c99012 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:24.655157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c05c4-2a54-44eb-97ac-f6e031c99012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:24.726114 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-912c05c4-2a54-44eb-97ac-f6e031c99012 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:24.727479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-912c05c4-2a54-44eb-97ac-f6e031c99012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:24.749042 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:24.830581 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 681/1375] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:24.876310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840f517-97fc-47b4-bae2-f629cbddd79e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:36:24.877958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8840f517-97fc-47b4-bae2-f629cbddd79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:25.033843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840f517-97fc-47b4-bae2-f629cbddd79e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:36:25.086608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:25.148977 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19948d9-56fc-44d3-bdad-01d866889e21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:36:25.149642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19948d9-56fc-44d3-bdad-01d866889e21 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:25.149959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19948d9-56fc-44d3-bdad-01d866889e21 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:25.150231 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19948d9-56fc-44d3-bdad-01d866889e21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:25.151949 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a19948d9-56fc-44d3-bdad-01d866889e21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:25.245480 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19948d9-56fc-44d3-bdad-01d866889e21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:36:25.247210 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19948d9-56fc-44d3-bdad-01d866889e21 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:36:25.271131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 695/1376] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:36:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:36:27.372353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d76c0a-11ae-4cbc-90cf-5acd834e4146 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:27.373001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d76c0a-11ae-4cbc-90cf-5acd834e4146 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:27.373001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d76c0a-11ae-4cbc-90cf-5acd834e4146 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:27.375619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 682/1377] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:36:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:27.380774 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff601a1-d67e-4dcd-981b-2d2d06052e6f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:27.381239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff601a1-d67e-4dcd-981b-2d2d06052e6f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:27.381342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff601a1-d67e-4dcd-981b-2d2d06052e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:27.450942 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6ff601a1-d67e-4dcd-981b-2d2d06052e6f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:27.452347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6ff601a1-d67e-4dcd-981b-2d2d06052e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:27.459217 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:27.561438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 696/1378] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:27.605384 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d76c0a-11ae-4cbc-90cf-5acd834e4146 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:36:27.607075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b3d76c0a-11ae-4cbc-90cf-5acd834e4146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:27.837416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d76c0a-11ae-4cbc-90cf-5acd834e4146 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:36:27.843370 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:27.982277 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66893c-ca72-4ad7-86e4-ae5452fca89d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:36:27.983832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66893c-ca72-4ad7-86e4-ae5452fca89d service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:27.984291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66893c-ca72-4ad7-86e4-ae5452fca89d service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:27.984715 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66893c-ca72-4ad7-86e4-ae5452fca89d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:27.986665 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-db66893c-ca72-4ad7-86e4-ae5452fca89d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:28.091612 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66893c-ca72-4ad7-86e4-ae5452fca89d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:36:28.094575 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66893c-ca72-4ad7-86e4-ae5452fca89d service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:36:28.114558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 683/1379] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:36:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:36:30.199899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb41b45-6749-452d-abb7-89a83aac3525 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:30.200447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb41b45-6749-452d-abb7-89a83aac3525 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:30.200447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb41b45-6749-452d-abb7-89a83aac3525 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:30.208250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d9d55-1f35-4ce5-b39c-997cf9ab9252 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:30.208777 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d9d55-1f35-4ce5-b39c-997cf9ab9252 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:30.208873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d9d55-1f35-4ce5-b39c-997cf9ab9252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:30.213779 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 697/1380] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:36:30 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:30.221668 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:30.299844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a88d9d55-1f35-4ce5-b39c-997cf9ab9252 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:30.301126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a88d9d55-1f35-4ce5-b39c-997cf9ab9252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:30.442637 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 684/1381] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:30.476281 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb41b45-6749-452d-abb7-89a83aac3525 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:36:30.477944 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6cb41b45-6749-452d-abb7-89a83aac3525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:30.640569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:30.658907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb41b45-6749-452d-abb7-89a83aac3525 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:36:30.813622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938febf3-8e7d-41ff-b5e5-ca663dae3e4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:36:30.814622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938febf3-8e7d-41ff-b5e5-ca663dae3e4e service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:30.815010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938febf3-8e7d-41ff-b5e5-ca663dae3e4e service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:30.815395 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938febf3-8e7d-41ff-b5e5-ca663dae3e4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:30.817254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-938febf3-8e7d-41ff-b5e5-ca663dae3e4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:30.923538 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938febf3-8e7d-41ff-b5e5-ca663dae3e4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:36:30.925223 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938febf3-8e7d-41ff-b5e5-ca663dae3e4e service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:36:30.937786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:30.948954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 698/1382] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:36:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:36:33.141686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58243d-1c11-4f95-b970-8a5e81b79f7b None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:33.142491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58243d-1c11-4f95-b970-8a5e81b79f7b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:33.142491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58243d-1c11-4f95-b970-8a5e81b79f7b None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:33.143589 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 685/1383] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:36:33.149837 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444e739-bbdd-443c-9976-62cb0ad54e60 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:33.150414 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444e739-bbdd-443c-9976-62cb0ad54e60 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:33.150654 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444e739-bbdd-443c-9976-62cb0ad54e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:33.250973 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b444e739-bbdd-443c-9976-62cb0ad54e60 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:33.252386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b444e739-bbdd-443c-9976-62cb0ad54e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:33.377937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 699/1384] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:33.426103 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58243d-1c11-4f95-b970-8a5e81b79f7b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:36:33.428097 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cc58243d-1c11-4f95-b970-8a5e81b79f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:33.541303 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:33.615876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58243d-1c11-4f95-b970-8a5e81b79f7b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:36:33.764895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5ae41-d535-425a-b495-f99efad076aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:36:33.766600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5ae41-d535-425a-b495-f99efad076aa service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:33.767508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5ae41-d535-425a-b495-f99efad076aa service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:33.768245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5ae41-d535-425a-b495-f99efad076aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:33.772124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a7d5ae41-d535-425a-b495-f99efad076aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:33.854722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:33.898388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5ae41-d535-425a-b495-f99efad076aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:36:33.900496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5ae41-d535-425a-b495-f99efad076aa service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:36:33.924794 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 686/1385] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:36:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:36:36.081009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84568af-3b9a-4831-a5aa-597570764e04 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:36.081765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84568af-3b9a-4831-a5aa-597570764e04 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:36.081765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84568af-3b9a-4831-a5aa-597570764e04 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:36.082989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 700/1386] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:36:36.088798 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f058aa-0b2b-4a13-aefd-383dc993334e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:36.089501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f058aa-0b2b-4a13-aefd-383dc993334e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:36.089501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f058aa-0b2b-4a13-aefd-383dc993334e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:36.163814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a5f058aa-0b2b-4a13-aefd-383dc993334e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:36.165026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a5f058aa-0b2b-4a13-aefd-383dc993334e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:36.301376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 687/1387] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:36.367385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84568af-3b9a-4831-a5aa-597570764e04 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:36:36.368931 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c84568af-3b9a-4831-a5aa-597570764e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:36.454074 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:36.537172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84568af-3b9a-4831-a5aa-597570764e04 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:36:36.673040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a18d33-7221-417f-87a0-645735911b3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:36:36.673595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a18d33-7221-417f-87a0-645735911b3e service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:36.673880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a18d33-7221-417f-87a0-645735911b3e service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:36.674155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a18d33-7221-417f-87a0-645735911b3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:36.675841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d8a18d33-7221-417f-87a0-645735911b3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:36.718465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:36.789347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a18d33-7221-417f-87a0-645735911b3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:36:36.791005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a18d33-7221-417f-87a0-645735911b3e service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:36:36.815822 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 701/1388] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:36:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:36:38.891744 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e2cd2-e433-448d-944e-69c9a31d9b6f None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:38.892313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e2cd2-e433-448d-944e-69c9a31d9b6f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:38.892313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e2cd2-e433-448d-944e-69c9a31d9b6f None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:38.893698 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 688/1389] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:38.899179 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbeae50-8385-4157-a1ce-8d878ae7bd7c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:38.899681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbeae50-8385-4157-a1ce-8d878ae7bd7c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:38.899786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbeae50-8385-4157-a1ce-8d878ae7bd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:38.970115 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9cbeae50-8385-4157-a1ce-8d878ae7bd7c None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:38.971290 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9cbeae50-8385-4157-a1ce-8d878ae7bd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:39.077640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 702/1390] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:39.124329 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e2cd2-e433-448d-944e-69c9a31d9b6f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:36:39.126102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ec7e2cd2-e433-448d-944e-69c9a31d9b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:39.164292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:39.318094 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e2cd2-e433-448d-944e-69c9a31d9b6f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:36:39.403855 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279caf7-d465-4af1-9b7f-58570e5dc87f None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:39.404198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279caf7-d465-4af1-9b7f-58570e5dc87f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:39.405124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279caf7-d465-4af1-9b7f-58570e5dc87f None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:39.406572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 703/1391] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:36:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:39.412077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200a883-5fe2-453c-b96f-85da98a3874d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:39.412373 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200a883-5fe2-453c-b96f-85da98a3874d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:39.412653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200a883-5fe2-453c-b96f-85da98a3874d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:39.428646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:39.437446 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:39.450998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b8ee5-e908-416f-acbb-759794d42edd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:36:39.452249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b8ee5-e908-416f-acbb-759794d42edd service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:39.452527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b8ee5-e908-416f-acbb-759794d42edd service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:39.452803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b8ee5-e908-416f-acbb-759794d42edd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:39.454628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5a3b8ee5-e908-416f-acbb-759794d42edd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:39.488514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6200a883-5fe2-453c-b96f-85da98a3874d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:39.489644 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6200a883-5fe2-453c-b96f-85da98a3874d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:39.556843 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b8ee5-e908-416f-acbb-759794d42edd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:36:39.558506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b8ee5-e908-416f-acbb-759794d42edd service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:36:39.578285 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 689/1392] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:36:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:36:39.590815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 704/1393] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:36:39.651184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981de42b-13dc-4fd7-b0ac-bb3197f3e1e2 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:39.651475 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981de42b-13dc-4fd7-b0ac-bb3197f3e1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:39.651758 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981de42b-13dc-4fd7-b0ac-bb3197f3e1e2 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:39.653249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 690/1394] 10.222.0.47 () {54 vars in 966 bytes} [Wed Aug 3 10:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:39.659820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac2d00-5c6f-409a-88a3-49834d441b1e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:39.660112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac2d00-5c6f-409a-88a3-49834d441b1e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:39.660388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac2d00-5c6f-409a-88a3-49834d441b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:39.749862 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-07ac2d00-5c6f-409a-88a3-49834d441b1e None None] MFA Rules not processed for user `300fb5e1c5c546808b935871673e3d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:39.751154 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-07ac2d00-5c6f-409a-88a3-49834d441b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:39.790316 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:39.835284 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 705/1395] 10.222.0.47 () {58 vars in 1063 bytes} [Wed Aug 3 10:36:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:39.843336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981de42b-13dc-4fd7-b0ac-bb3197f3e1e2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:36:39.845188 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-981de42b-13dc-4fd7-b0ac-bb3197f3e1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:39.961295 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981de42b-13dc-4fd7-b0ac-bb3197f3e1e2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:36:39.977386 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:40.041100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c59864-ff59-4246-9410-6296db42982b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '300fb5e1c5c546808b935871673e3d0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:36:40.041767 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c59864-ff59-4246-9410-6296db42982b service glance] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:40.042055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c59864-ff59-4246-9410-6296db42982b service glance] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:40.042328 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c59864-ff59-4246-9410-6296db42982b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:40.043945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-40c59864-ff59-4246-9410-6296db42982b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:40.156415 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c59864-ff59-4246-9410-6296db42982b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:36:40.157791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c59864-ff59-4246-9410-6296db42982b service glance] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:36:40.176274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 691/1396] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:36:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:36:40.932208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a27b14-ad55-445d-b76b-6887343c3d88 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:40.932696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a27b14-ad55-445d-b76b-6887343c3d88 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:40.932791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a27b14-ad55-445d-b76b-6887343c3d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:41.003118 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-70a27b14-ad55-445d-b76b-6887343c3d88 None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:41.004325 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-70a27b14-ad55-445d-b76b-6887343c3d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:41.106420 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 706/1397] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 3 10:36:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:41.150771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d57f1b-765c-41c1-9fdb-7dd4e0946efe None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:41.151278 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d57f1b-765c-41c1-9fdb-7dd4e0946efe None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:41.151373 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d57f1b-765c-41c1-9fdb-7dd4e0946efe None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:41.152882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 692/1398] 10.222.0.47 () {54 vars in 943 bytes} [Wed Aug 3 10:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:41.160036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c4c5a-9e29-456b-919b-e9fe667b6aab None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:41.160329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c4c5a-9e29-456b-919b-e9fe667b6aab None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:41.160638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c4c5a-9e29-456b-919b-e9fe667b6aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:41.222218 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:41.244516 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-752c4c5a-9e29-456b-919b-e9fe667b6aab None None] MFA Rules not processed for user `a81ea5afe74b43879abc94b3cd134fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:41.245759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-752c4c5a-9e29-456b-919b-e9fe667b6aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:41.356325 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 707/1399] 10.222.0.47 () {58 vars in 1040 bytes} [Wed Aug 3 10:36:41 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:41.366044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d57f1b-765c-41c1-9fdb-7dd4e0946efe None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:36:41.367557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-70d57f1b-765c-41c1-9fdb-7dd4e0946efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:41.416367 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:41.516352 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d57f1b-765c-41c1-9fdb-7dd4e0946efe None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:36:41.634042 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acf28a-f388-4ceb-88c8-8ae93d4aa6a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:36:41.634825 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acf28a-f388-4ceb-88c8-8ae93d4aa6a9 service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:41.635110 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acf28a-f388-4ceb-88c8-8ae93d4aa6a9 service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:41.635389 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acf28a-f388-4ceb-88c8-8ae93d4aa6a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:41.637123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-25acf28a-f388-4ceb-88c8-8ae93d4aa6a9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:41.640471 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27762cd4-a39d-4e57-a9a4-476c3f2a39ed None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:41.640798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27762cd4-a39d-4e57-a9a4-476c3f2a39ed None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:41.641077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27762cd4-a39d-4e57-a9a4-476c3f2a39ed None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:41.642489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 708/1400] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:36:41.648663 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14827dca-9d2e-4534-ae33-0fed51f78f71 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:41.649026 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14827dca-9d2e-4534-ae33-0fed51f78f71 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:41.649327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14827dca-9d2e-4534-ae33-0fed51f78f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:41.678470 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:41.720550 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-14827dca-9d2e-4534-ae33-0fed51f78f71 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:41.721910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-14827dca-9d2e-4534-ae33-0fed51f78f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:41.739529 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acf28a-f388-4ceb-88c8-8ae93d4aa6a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:36:41.741134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acf28a-f388-4ceb-88c8-8ae93d4aa6a9 service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:36:41.750895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 693/1401] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:36:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 10:36:41.782582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:41.829598 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 709/1402] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:36:41.879310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acf28a-f388-4ceb-88c8-8ae93d4aa6a9 service swift] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:36:41.880547 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-25acf28a-f388-4ceb-88c8-8ae93d4aa6a9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:41.994345 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acf28a-f388-4ceb-88c8-8ae93d4aa6a9 service swift] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:36:42.086453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:42.141358 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070e3c1-f876-4f72-afb4-7e16a0853dbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:36:42.142042 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070e3c1-f876-4f72-afb4-7e16a0853dbb service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:42.142327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070e3c1-f876-4f72-afb4-7e16a0853dbb service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:42.142596 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070e3c1-f876-4f72-afb4-7e16a0853dbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:42.144421 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3070e3c1-f876-4f72-afb4-7e16a0853dbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:42.261733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070e3c1-f876-4f72-afb4-7e16a0853dbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:36:42.263387 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070e3c1-f876-4f72-afb4-7e16a0853dbb service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:36:42.293701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 694/1403] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:36:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:36:44.396313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf6de0-4b8b-47fd-a2e0-e67f3841b2d1 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:44.396912 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf6de0-4b8b-47fd-a2e0-e67f3841b2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:44.397282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf6de0-4b8b-47fd-a2e0-e67f3841b2d1 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:44.399825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 710/1404] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:36:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:44.406517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52dc53-1e3d-4371-9004-64bbbdfccdcf None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:44.407068 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52dc53-1e3d-4371-9004-64bbbdfccdcf None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:44.407068 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52dc53-1e3d-4371-9004-64bbbdfccdcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:44.483768 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2a52dc53-1e3d-4371-9004-64bbbdfccdcf None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:44.485095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2a52dc53-1e3d-4371-9004-64bbbdfccdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:44.524986 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:44.614967 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 695/1405] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:36:44.666028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf6de0-4b8b-47fd-a2e0-e67f3841b2d1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:36:44.667569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7aaf6de0-4b8b-47fd-a2e0-e67f3841b2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:44.868421 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf6de0-4b8b-47fd-a2e0-e67f3841b2d1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:36:44.918175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:45.019079 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386c415f-a0fb-4fc6-af5f-da6b2dc72a1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:36:45.019755 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c415f-a0fb-4fc6-af5f-da6b2dc72a1c service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:45.020034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c415f-a0fb-4fc6-af5f-da6b2dc72a1c service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:45.020326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c415f-a0fb-4fc6-af5f-da6b2dc72a1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:45.022054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-386c415f-a0fb-4fc6-af5f-da6b2dc72a1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:45.145061 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c415f-a0fb-4fc6-af5f-da6b2dc72a1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:36:45.149007 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c415f-a0fb-4fc6-af5f-da6b2dc72a1c service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:36:45.176888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 711/1406] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:36:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:36:47.322004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e15829-7bbe-4d86-af16-85a69a5796df None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:47.322571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e15829-7bbe-4d86-af16-85a69a5796df None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:47.322571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e15829-7bbe-4d86-af16-85a69a5796df None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:47.324047 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 696/1407] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:36:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:36:47.330340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb8f5e-bf3e-437a-af40-58a57637f5b7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:47.330901 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb8f5e-bf3e-437a-af40-58a57637f5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:47.330901 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb8f5e-bf3e-437a-af40-58a57637f5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:47.399614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:47.412350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-bfdb8f5e-bf3e-437a-af40-58a57637f5b7 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:36:47.413765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bfdb8f5e-bf3e-437a-af40-58a57637f5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:47.566385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 712/1408] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:36:47.622018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e15829-7bbe-4d86-af16-85a69a5796df None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:36:47.623594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-99e15829-7bbe-4d86-af16-85a69a5796df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:47.799811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e15829-7bbe-4d86-af16-85a69a5796df None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:36:47.821717 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:47.941867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf4038-d769-4d1c-94f4-8a766921875f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:36:47.943415 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf4038-d769-4d1c-94f4-8a766921875f service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:36:47.943793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf4038-d769-4d1c-94f4-8a766921875f service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:36:47.944171 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf4038-d769-4d1c-94f4-8a766921875f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:36:47.945986 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bedf4038-d769-4d1c-94f4-8a766921875f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:36:48.053341 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf4038-d769-4d1c-94f4-8a766921875f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:36:48.055055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf4038-d769-4d1c-94f4-8a766921875f service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:36:48.078219 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 697/1409] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:36:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:36:57.563664 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=44 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:36:58.075224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:04.438140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056a897-6128-4950-966c-79d7f77bd08b None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:04.438717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056a897-6128-4950-966c-79d7f77bd08b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:04.438717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056a897-6128-4950-966c-79d7f77bd08b None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:04.440068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 713/1410] 10.222.0.47 () {54 vars in 918 bytes} [Wed Aug 3 10:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:04.445955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4147b7-aff2-4def-a3f9-34a25584a3b9 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:04.446443 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4147b7-aff2-4def-a3f9-34a25584a3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:04.446541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4147b7-aff2-4def-a3f9-34a25584a3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:04.524051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0c4147b7-aff2-4def-a3f9-34a25584a3b9 None None] MFA Rules not processed for user `f2c5536eddd346d692822a2f3befb48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:04.525261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0c4147b7-aff2-4def-a3f9-34a25584a3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:04.641787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 698/1411] 10.222.0.47 () {58 vars in 1015 bytes} [Wed Aug 3 10:37:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:04.657061 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f7d1f-a83c-492c-b83f-f8aa68dc911a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:04.657345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f7d1f-a83c-492c-b83f-f8aa68dc911a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:04.657653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f7d1f-a83c-492c-b83f-f8aa68dc911a None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:04.659012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 714/1412] 10.222.0.47 () {54 vars in 918 bytes} [Wed Aug 3 10:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:04.664172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ba835-b3a7-4215-ae00-b27be5485874 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:04.664468 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ba835-b3a7-4215-ae00-b27be5485874 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:04.664750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ba835-b3a7-4215-ae00-b27be5485874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:04.748128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:04.752177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-cf6ba835-b3a7-4215-ae00-b27be5485874 None None] MFA Rules not processed for user `e66f25cc329f418eba6bd637bf3e8a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:04.754265 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cf6ba835-b3a7-4215-ae00-b27be5485874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:04.874187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 699/1413] 10.222.0.47 () {58 vars in 1015 bytes} [Wed Aug 3 10:37:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:07.375841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8038c7-8d5a-4fb8-88a4-d33e94f17650 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:07.376385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8038c7-8d5a-4fb8-88a4-d33e94f17650 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:07.376385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8038c7-8d5a-4fb8-88a4-d33e94f17650 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:07.377877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 10:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:07.383601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e03f54-00f1-41cb-9d9c-8d51c8831d47 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:07.384156 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e03f54-00f1-41cb-9d9c-8d51c8831d47 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:07.384156 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e03f54-00f1-41cb-9d9c-8d51c8831d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:07.455444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-29e03f54-00f1-41cb-9d9c-8d51c8831d47 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:07.456574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-29e03f54-00f1-41cb-9d9c-8d51c8831d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:07.512882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:07.556933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 10:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:07.600347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8038c7-8d5a-4fb8-88a4-d33e94f17650 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:07.602010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0b8038c7-8d5a-4fb8-88a4-d33e94f17650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:07.754635 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8038c7-8d5a-4fb8-88a4-d33e94f17650 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:07.827897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:07.869232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12eb47-9313-4723-8072-b62925fe14d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:07.869882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12eb47-9313-4723-8072-b62925fe14d6 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:07.870164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12eb47-9313-4723-8072-b62925fe14d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:07.870434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12eb47-9313-4723-8072-b62925fe14d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:07.872086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7d12eb47-9313-4723-8072-b62925fe14d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:07.960340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12eb47-9313-4723-8072-b62925fe14d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:07.961981 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12eb47-9313-4723-8072-b62925fe14d6 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:07.980876 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 716/1416] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:08.892307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40d6e5-3017-4877-84a4-a95d80782ee9 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:08.892849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40d6e5-3017-4877-84a4-a95d80782ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:08.892849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40d6e5-3017-4877-84a4-a95d80782ee9 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:08.894291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 10:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:08.899315 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d6d4e-5c18-42b3-9d68-1aa1188341b6 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:08.899758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d6d4e-5c18-42b3-9d68-1aa1188341b6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:08.899879 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d6d4e-5c18-42b3-9d68-1aa1188341b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:08.977142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4b0d6d4e-5c18-42b3-9d68-1aa1188341b6 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:08.978415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4b0d6d4e-5c18-42b3-9d68-1aa1188341b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:09.006703 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:09.078163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:09.229783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40d6e5-3017-4877-84a4-a95d80782ee9 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:09.231332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7b40d6e5-3017-4877-84a4-a95d80782ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:09.384212 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40d6e5-3017-4877-84a4-a95d80782ee9 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:09.432023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:09.509712 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2a415-3857-487b-b407-8c70a888c6ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:09.510248 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2a415-3857-487b-b407-8c70a888c6ec service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:09.510527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2a415-3857-487b-b407-8c70a888c6ec service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:09.510799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2a415-3857-487b-b407-8c70a888c6ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:09.512424 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9df2a415-3857-487b-b407-8c70a888c6ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:09.602669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2a415-3857-487b-b407-8c70a888c6ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:09.604275 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2a415-3857-487b-b407-8c70a888c6ec service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:09.622103 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 702/1419] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:10.519452 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c7009-bcdd-4786-a28c-ab45ba278158 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:10.520031 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c7009-bcdd-4786-a28c-ab45ba278158 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:10.520031 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c7009-bcdd-4786-a28c-ab45ba278158 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:10.521378 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 10:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:10.526721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3205970-83d5-4a8e-a0b8-5a70650d085a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:10.527184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3205970-83d5-4a8e-a0b8-5a70650d085a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:10.527287 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3205970-83d5-4a8e-a0b8-5a70650d085a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:10.582915 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:10.609919 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e3205970-83d5-4a8e-a0b8-5a70650d085a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:10.611134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e3205970-83d5-4a8e-a0b8-5a70650d085a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:10.714599 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 10:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:10.872992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c7009-bcdd-4786-a28c-ab45ba278158 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:10.874524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-654c7009-bcdd-4786-a28c-ab45ba278158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:11.022753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c7009-bcdd-4786-a28c-ab45ba278158 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:11.036464 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:11.144454 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5912e5-1887-46f7-9522-19b22c6789c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:11.144985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5912e5-1887-46f7-9522-19b22c6789c8 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:11.145263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5912e5-1887-46f7-9522-19b22c6789c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:11.145566 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5912e5-1887-46f7-9522-19b22c6789c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:11.147203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0d5912e5-1887-46f7-9522-19b22c6789c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:11.236701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5912e5-1887-46f7-9522-19b22c6789c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:11.238435 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5912e5-1887-46f7-9522-19b22c6789c8 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:11.256954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 719/1422] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:12.136314 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca7e0e-a403-4a77-82a0-fc4558134d72 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:12.136892 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca7e0e-a403-4a77-82a0-fc4558134d72 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:12.136892 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca7e0e-a403-4a77-82a0-fc4558134d72 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:12.138273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 10:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:12.143295 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0d41f-1e24-498d-8068-0c6d34dc283f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:12.143752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0d41f-1e24-498d-8068-0c6d34dc283f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:12.143862 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0d41f-1e24-498d-8068-0c6d34dc283f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:12.163956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:12.221304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-deb0d41f-1e24-498d-8068-0c6d34dc283f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:12.222586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-deb0d41f-1e24-498d-8068-0c6d34dc283f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:12.319307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 10:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:12.470909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca7e0e-a403-4a77-82a0-fc4558134d72 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:12.472415 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d5ca7e0e-a403-4a77-82a0-fc4558134d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:12.583250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:12.623411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca7e0e-a403-4a77-82a0-fc4558134d72 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:12.747376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e4758-e5d5-48a5-98e7-8f5f9ff92b89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:12.747917 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e4758-e5d5-48a5-98e7-8f5f9ff92b89 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:12.748200 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e4758-e5d5-48a5-98e7-8f5f9ff92b89 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:12.749126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e4758-e5d5-48a5-98e7-8f5f9ff92b89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:12.751444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-432e4758-e5d5-48a5-98e7-8f5f9ff92b89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:12.831336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:12.845398 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e4758-e5d5-48a5-98e7-8f5f9ff92b89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:12.847287 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e4758-e5d5-48a5-98e7-8f5f9ff92b89 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:12.865808 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 705/1425] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:13.746858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0976d27-ea02-45c0-8697-90fcdbc2fea0 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:13.747473 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0976d27-ea02-45c0-8697-90fcdbc2fea0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:13.747473 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0976d27-ea02-45c0-8697-90fcdbc2fea0 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:13.748780 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 10:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:13.753876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8553733-2191-4b4d-9ced-867c37c4ceec None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:13.754387 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8553733-2191-4b4d-9ced-867c37c4ceec None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:13.754508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8553733-2191-4b4d-9ced-867c37c4ceec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:13.832641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d8553733-2191-4b4d-9ced-867c37c4ceec None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:13.833913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d8553733-2191-4b4d-9ced-867c37c4ceec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:13.935140 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 10:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:14.083837 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0976d27-ea02-45c0-8697-90fcdbc2fea0 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:14.085356 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a0976d27-ea02-45c0-8697-90fcdbc2fea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:14.168188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:14.230782 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0976d27-ea02-45c0-8697-90fcdbc2fea0 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:14.344056 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0e6e9-236f-4041-aff4-f1d886098a52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:14.344679 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0e6e9-236f-4041-aff4-f1d886098a52 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:14.344961 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0e6e9-236f-4041-aff4-f1d886098a52 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:14.345234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0e6e9-236f-4041-aff4-f1d886098a52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:14.347220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f1e0e6e9-236f-4041-aff4-f1d886098a52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:14.397227 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:14.441220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0e6e9-236f-4041-aff4-f1d886098a52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:14.442776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0e6e9-236f-4041-aff4-f1d886098a52 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:14.460594 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 722/1428] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:15.279431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f4de5-b602-471d-9a3b-5b213113e238 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:15.279856 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f4de5-b602-471d-9a3b-5b213113e238 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:15.279994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f4de5-b602-471d-9a3b-5b213113e238 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:15.281440 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 10:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:15.286731 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27e51a-654d-43c1-8863-bac97257a2e4 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:15.287185 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27e51a-654d-43c1-8863-bac97257a2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:15.287292 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27e51a-654d-43c1-8863-bac97257a2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:15.366459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2d27e51a-654d-43c1-8863-bac97257a2e4 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:15.367695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2d27e51a-654d-43c1-8863-bac97257a2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:15.464501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 10:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:15.613914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f4de5-b602-471d-9a3b-5b213113e238 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:15.615435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8b7f4de5-b602-471d-9a3b-5b213113e238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:15.664947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:15.765525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f4de5-b602-471d-9a3b-5b213113e238 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:15.882356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9c54e-5d6e-4c15-9b8c-2dea37f33e75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:15.883010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9c54e-5d6e-4c15-9b8c-2dea37f33e75 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:15.883295 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9c54e-5d6e-4c15-9b8c-2dea37f33e75 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:15.883571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9c54e-5d6e-4c15-9b8c-2dea37f33e75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:15.885203 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d4a9c54e-5d6e-4c15-9b8c-2dea37f33e75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:15.892232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:15.984190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9c54e-5d6e-4c15-9b8c-2dea37f33e75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:15.985812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9c54e-5d6e-4c15-9b8c-2dea37f33e75 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:16.011662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 708/1431] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:17.002691 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e08b17-2dcb-4b7c-95e3-49fe1a0ccfd9 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:17.003254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e08b17-2dcb-4b7c-95e3-49fe1a0ccfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:17.003254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e08b17-2dcb-4b7c-95e3-49fe1a0ccfd9 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:17.004665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 10:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:17.010068 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faad57c-e473-4f99-8e15-884db8418a4b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:17.010540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faad57c-e473-4f99-8e15-884db8418a4b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:17.010644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faad57c-e473-4f99-8e15-884db8418a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:17.090018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-9faad57c-e473-4f99-8e15-884db8418a4b None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:17.091636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9faad57c-e473-4f99-8e15-884db8418a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:17.166436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:17.196835 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 10:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:17.355875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e08b17-2dcb-4b7c-95e3-49fe1a0ccfd9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:17.357426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-93e08b17-2dcb-4b7c-95e3-49fe1a0ccfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:17.375581 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:17.506330 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e08b17-2dcb-4b7c-95e3-49fe1a0ccfd9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:17.598169 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:17.624533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de123db9-0146-45e7-855e-33fa40d51978 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:17.625696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de123db9-0146-45e7-855e-33fa40d51978 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:17.625696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de123db9-0146-45e7-855e-33fa40d51978 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:17.625927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de123db9-0146-45e7-855e-33fa40d51978 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:17.627391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-de123db9-0146-45e7-855e-33fa40d51978 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:17.722875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de123db9-0146-45e7-855e-33fa40d51978 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:17.724484 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de123db9-0146-45e7-855e-33fa40d51978 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:17.743003 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 725/1434] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:18.546081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395fa28-6f1d-4119-9011-b5f3a506d9b3 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:18.546640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395fa28-6f1d-4119-9011-b5f3a506d9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:18.546640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395fa28-6f1d-4119-9011-b5f3a506d9b3 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:18.548014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 10:37:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:18.553056 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6c575-c05a-420f-8471-94241f986a3a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:18.553546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6c575-c05a-420f-8471-94241f986a3a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:18.553648 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6c575-c05a-420f-8471-94241f986a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:18.634117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-87d6c575-c05a-420f-8471-94241f986a3a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:18.635342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-87d6c575-c05a-420f-8471-94241f986a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:18.676913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:18.731714 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 10:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:18.883409 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395fa28-6f1d-4119-9011-b5f3a506d9b3 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:18.884832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c395fa28-6f1d-4119-9011-b5f3a506d9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:19.028272 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395fa28-6f1d-4119-9011-b5f3a506d9b3 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:19.083091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:19.143718 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f2c0e-0962-4ba2-841e-4ec6ed7cf4aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:19.144267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f2c0e-0962-4ba2-841e-4ec6ed7cf4aa service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:19.144552 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f2c0e-0962-4ba2-841e-4ec6ed7cf4aa service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:19.144831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f2c0e-0962-4ba2-841e-4ec6ed7cf4aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:19.146502 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d76f2c0e-0962-4ba2-841e-4ec6ed7cf4aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:19.242840 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f2c0e-0962-4ba2-841e-4ec6ed7cf4aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:19.244471 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f2c0e-0962-4ba2-841e-4ec6ed7cf4aa service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:19.262309 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 711/1437] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:20.071787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f81946-89bf-4c52-8342-f5cb69a872a3 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:20.072358 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f81946-89bf-4c52-8342-f5cb69a872a3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:20.072358 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f81946-89bf-4c52-8342-f5cb69a872a3 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:20.073729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 10:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:20.079265 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76d8dd-f20d-4292-b636-d4c8c193e038 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:20.079677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76d8dd-f20d-4292-b636-d4c8c193e038 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:20.079830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76d8dd-f20d-4292-b636-d4c8c193e038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:20.157106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5f76d8dd-f20d-4292-b636-d4c8c193e038 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:20.158357 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5f76d8dd-f20d-4292-b636-d4c8c193e038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:20.160384 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:20.255726 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 10:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:20.414510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f81946-89bf-4c52-8342-f5cb69a872a3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:20.416034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c1f81946-89bf-4c52-8342-f5cb69a872a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:20.560670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f81946-89bf-4c52-8342-f5cb69a872a3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:20.587902 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:20.677301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f522c-0b14-4728-8382-3c7b292e2d3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:20.677954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f522c-0b14-4728-8382-3c7b292e2d3a service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:20.678255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f522c-0b14-4728-8382-3c7b292e2d3a service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:20.678527 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f522c-0b14-4728-8382-3c7b292e2d3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:20.680185 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-140f522c-0b14-4728-8382-3c7b292e2d3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:20.770643 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f522c-0b14-4728-8382-3c7b292e2d3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:20.772237 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f522c-0b14-4728-8382-3c7b292e2d3a service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:20.790375 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 728/1440] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:22.700322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74301f7a-6b7b-4087-a4d6-77d6d142ce7a None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:22.700830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74301f7a-6b7b-4087-a4d6-77d6d142ce7a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:22.700964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74301f7a-6b7b-4087-a4d6-77d6d142ce7a None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:22.702318 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 10:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:22.708030 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bebe3-ca11-41c2-9af4-e01d5632fe6d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:22.708436 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bebe3-ca11-41c2-9af4-e01d5632fe6d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:22.708590 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bebe3-ca11-41c2-9af4-e01d5632fe6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:22.751040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:22.780782 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-428bebe3-ca11-41c2-9af4-e01d5632fe6d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:22.782154 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-428bebe3-ca11-41c2-9af4-e01d5632fe6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:22.882584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 10:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:22.921669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74301f7a-6b7b-4087-a4d6-77d6d142ce7a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:22.923245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-74301f7a-6b7b-4087-a4d6-77d6d142ce7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:23.028167 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74301f7a-6b7b-4087-a4d6-77d6d142ce7a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:23.060621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:23.102577 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febcca6b-ddb9-436a-b39a-4a4cc0f8b847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:23.103209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febcca6b-ddb9-436a-b39a-4a4cc0f8b847 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:23.103496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febcca6b-ddb9-436a-b39a-4a4cc0f8b847 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:23.103774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febcca6b-ddb9-436a-b39a-4a4cc0f8b847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:23.105445 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-febcca6b-ddb9-436a-b39a-4a4cc0f8b847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:23.225798 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febcca6b-ddb9-436a-b39a-4a4cc0f8b847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:23.227241 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febcca6b-ddb9-436a-b39a-4a4cc0f8b847 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:23.246495 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 714/1443] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:25.411329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea86900-d15b-44fa-8dd5-06b89cdb410d None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:25.411915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea86900-d15b-44fa-8dd5-06b89cdb410d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:25.411915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea86900-d15b-44fa-8dd5-06b89cdb410d None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:25.413291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 10:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:25.418609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097aea6-8359-4f6a-99eb-1b256d3abbd5 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:25.419196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097aea6-8359-4f6a-99eb-1b256d3abbd5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:25.419196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097aea6-8359-4f6a-99eb-1b256d3abbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:25.487267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f097aea6-8359-4f6a-99eb-1b256d3abbd5 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:25.488408 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f097aea6-8359-4f6a-99eb-1b256d3abbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:25.501719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:25.587845 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 10:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:25.626387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea86900-d15b-44fa-8dd5-06b89cdb410d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:25.627948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3ea86900-d15b-44fa-8dd5-06b89cdb410d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:25.732769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea86900-d15b-44fa-8dd5-06b89cdb410d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:25.745157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:25.809255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fc7e4-f6cc-4610-a40a-d91af7f2d844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:25.809900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fc7e4-f6cc-4610-a40a-d91af7f2d844 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:25.810178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fc7e4-f6cc-4610-a40a-d91af7f2d844 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:25.810449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fc7e4-f6cc-4610-a40a-d91af7f2d844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:25.812589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-434fc7e4-f6cc-4610-a40a-d91af7f2d844 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:25.920786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fc7e4-f6cc-4610-a40a-d91af7f2d844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:25.922248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fc7e4-f6cc-4610-a40a-d91af7f2d844 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:25.940494 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 731/1446] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:28.281444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ec1ed-c1a4-4599-b83a-a3f39b4f4eaf None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:28.282030 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ec1ed-c1a4-4599-b83a-a3f39b4f4eaf None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:28.282030 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ec1ed-c1a4-4599-b83a-a3f39b4f4eaf None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:28.283438 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 10:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:28.288899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ad8a6-5b29-481b-b5ed-ff283c96b852 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:28.289346 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ad8a6-5b29-481b-b5ed-ff283c96b852 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:28.289450 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ad8a6-5b29-481b-b5ed-ff283c96b852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:28.338265 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:28.357722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-756ad8a6-5b29-481b-b5ed-ff283c96b852 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:28.358950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-756ad8a6-5b29-481b-b5ed-ff283c96b852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:28.459287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 10:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:28.499368 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ec1ed-c1a4-4599-b83a-a3f39b4f4eaf None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:28.500927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-781ec1ed-c1a4-4599-b83a-a3f39b4f4eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:28.605402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ec1ed-c1a4-4599-b83a-a3f39b4f4eaf None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:28.645049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:28.681882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbe61e-85f9-4abf-8034-114efa91465b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:28.682528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbe61e-85f9-4abf-8034-114efa91465b service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:28.682878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbe61e-85f9-4abf-8034-114efa91465b service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:28.683089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbe61e-85f9-4abf-8034-114efa91465b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:28.685230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ddfbe61e-85f9-4abf-8034-114efa91465b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:28.798862 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbe61e-85f9-4abf-8034-114efa91465b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:28.800306 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbe61e-85f9-4abf-8034-114efa91465b service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:28.818264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 717/1449] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:31.024273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a39cf5-4da5-4210-85ed-dd19c700c44d None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:31.024810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a39cf5-4da5-4210-85ed-dd19c700c44d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:31.024810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a39cf5-4da5-4210-85ed-dd19c700c44d None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:31.026289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 10:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:31.032203 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60989d1-5ca8-4dfb-ac64-eb43cc090650 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:31.032637 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60989d1-5ca8-4dfb-ac64-eb43cc090650 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:31.032747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60989d1-5ca8-4dfb-ac64-eb43cc090650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:31.105439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c60989d1-5ca8-4dfb-ac64-eb43cc090650 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:31.106703 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c60989d1-5ca8-4dfb-ac64-eb43cc090650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:31.127310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:31.206953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 10:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:31.246533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a39cf5-4da5-4210-85ed-dd19c700c44d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:31.248109 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-65a39cf5-4da5-4210-85ed-dd19c700c44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:31.350924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a39cf5-4da5-4210-85ed-dd19c700c44d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:31.370312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:31.424606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034a595-98ec-4afd-a1a6-e693d0d0271e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:31.425223 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034a595-98ec-4afd-a1a6-e693d0d0271e service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:31.425526 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034a595-98ec-4afd-a1a6-e693d0d0271e service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:31.425802 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034a595-98ec-4afd-a1a6-e693d0d0271e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:31.427432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5034a595-98ec-4afd-a1a6-e693d0d0271e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:31.538605 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034a595-98ec-4afd-a1a6-e693d0d0271e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:31.540027 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034a595-98ec-4afd-a1a6-e693d0d0271e service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:31.558447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 734/1452] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:33.627691 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebaf40-e063-4cb9-9874-169deb800153 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:33.628213 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebaf40-e063-4cb9-9874-169deb800153 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:33.628213 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebaf40-e063-4cb9-9874-169deb800153 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:33.629554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 10:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:33.635366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef95cf-de1e-4d5a-b2b5-f01f74156a4e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:33.635829 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef95cf-de1e-4d5a-b2b5-f01f74156a4e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:33.635927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef95cf-de1e-4d5a-b2b5-f01f74156a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:33.705295 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:33.708246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b8ef95cf-de1e-4d5a-b2b5-f01f74156a4e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:33.710069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b8ef95cf-de1e-4d5a-b2b5-f01f74156a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:33.804562 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 10:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:33.842742 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebaf40-e063-4cb9-9874-169deb800153 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:33.844324 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-01ebaf40-e063-4cb9-9874-169deb800153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:33.950433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebaf40-e063-4cb9-9874-169deb800153 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:33.999295 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:34.024240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5eeb0-8881-4978-9946-dc863ed5accd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:34.024865 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5eeb0-8881-4978-9946-dc863ed5accd service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:34.025146 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5eeb0-8881-4978-9946-dc863ed5accd service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:34.025422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5eeb0-8881-4978-9946-dc863ed5accd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:34.027073 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8ce5eeb0-8881-4978-9946-dc863ed5accd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:34.137033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5eeb0-8881-4978-9946-dc863ed5accd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:34.138544 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5eeb0-8881-4978-9946-dc863ed5accd service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:34.156800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 720/1455] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:36.259970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c864d-d00e-404f-bdaa-b1dec7f82f32 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:36.260542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c864d-d00e-404f-bdaa-b1dec7f82f32 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:36.260542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c864d-d00e-404f-bdaa-b1dec7f82f32 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:36.261958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 10:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:36.267269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae8ae4-620e-4e1e-a980-8b5c9c4e769f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:36.267697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae8ae4-620e-4e1e-a980-8b5c9c4e769f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:36.267825 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae8ae4-620e-4e1e-a980-8b5c9c4e769f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:36.336110 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b7ae8ae4-620e-4e1e-a980-8b5c9c4e769f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:36.337361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b7ae8ae4-620e-4e1e-a980-8b5c9c4e769f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:36.366662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:36.434027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 10:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:36.472937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c864d-d00e-404f-bdaa-b1dec7f82f32 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:36.474671 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-be1c864d-d00e-404f-bdaa-b1dec7f82f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:36.580615 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c864d-d00e-404f-bdaa-b1dec7f82f32 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:36.615456 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:36.661392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e39e4-3b72-420c-93a9-624751118c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:36.662040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e39e4-3b72-420c-93a9-624751118c5f service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:36.662321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e39e4-3b72-420c-93a9-624751118c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:36.662590 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e39e4-3b72-420c-93a9-624751118c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:36.664206 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a66e39e4-3b72-420c-93a9-624751118c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:36.773197 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e39e4-3b72-420c-93a9-624751118c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:36.774656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e39e4-3b72-420c-93a9-624751118c5f service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:36.792382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 737/1458] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:38.855111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663a167-ac33-4a38-9c7a-fd010862dc0e None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:38.855641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663a167-ac33-4a38-9c7a-fd010862dc0e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:38.855641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663a167-ac33-4a38-9c7a-fd010862dc0e None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:38.857105 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 10:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:38.863132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f834732-18cd-4972-b928-3fca71c57752 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:38.863574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f834732-18cd-4972-b928-3fca71c57752 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:38.863690 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f834732-18cd-4972-b928-3fca71c57752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:38.930742 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8f834732-18cd-4972-b928-3fca71c57752 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:38.931969 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8f834732-18cd-4972-b928-3fca71c57752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:38.941226 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:39.032998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 10:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:37:39.070447 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663a167-ac33-4a38-9c7a-fd010862dc0e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:39.072016 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d663a167-ac33-4a38-9c7a-fd010862dc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:39.175671 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663a167-ac33-4a38-9c7a-fd010862dc0e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:39.245678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:39.263818 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95ac3f-5840-41d1-9bfc-cac610dbe1ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:39.264447 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95ac3f-5840-41d1-9bfc-cac610dbe1ef service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:39.264742 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95ac3f-5840-41d1-9bfc-cac610dbe1ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:39.265011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95ac3f-5840-41d1-9bfc-cac610dbe1ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:39.266684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1e95ac3f-5840-41d1-9bfc-cac610dbe1ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:39.376148 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95ac3f-5840-41d1-9bfc-cac610dbe1ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:39.377662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95ac3f-5840-41d1-9bfc-cac610dbe1ef service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:39.395738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 723/1461] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:41.469437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b661cfd-8fbe-4d1d-8c65-27af53372f59 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:41.470050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b661cfd-8fbe-4d1d-8c65-27af53372f59 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:41.470050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b661cfd-8fbe-4d1d-8c65-27af53372f59 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:41.471399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 10:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:41.477293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f532a1d-67ff-4203-992b-1b0308c1d570 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:41.477814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f532a1d-67ff-4203-992b-1b0308c1d570 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:41.477914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f532a1d-67ff-4203-992b-1b0308c1d570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:41.544013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3f532a1d-67ff-4203-992b-1b0308c1d570 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:41.545262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3f532a1d-67ff-4203-992b-1b0308c1d570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:41.584733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:41.652170 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 10:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:37:41.689849 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b661cfd-8fbe-4d1d-8c65-27af53372f59 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:41.691413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3b661cfd-8fbe-4d1d-8c65-27af53372f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:41.796036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b661cfd-8fbe-4d1d-8c65-27af53372f59 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:41.833417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:41.869908 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad0a80-cc13-4b21-9f5f-d2cc8c3e5cc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:41.870531 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad0a80-cc13-4b21-9f5f-d2cc8c3e5cc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:41.870897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad0a80-cc13-4b21-9f5f-d2cc8c3e5cc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:41.871090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad0a80-cc13-4b21-9f5f-d2cc8c3e5cc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:41.872720 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-88ad0a80-cc13-4b21-9f5f-d2cc8c3e5cc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:41.982203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad0a80-cc13-4b21-9f5f-d2cc8c3e5cc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:41.984202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad0a80-cc13-4b21-9f5f-d2cc8c3e5cc0 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:42.002665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 740/1464] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:44.045585 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0ab9b-4453-457c-8c50-11f2b5998f2d None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:44.046241 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0ab9b-4453-457c-8c50-11f2b5998f2d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:44.046241 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0ab9b-4453-457c-8c50-11f2b5998f2d None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:44.047506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 10:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:44.053205 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af542d2-61b7-4e9f-8707-b7f72e910dd0 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:44.053758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af542d2-61b7-4e9f-8707-b7f72e910dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:44.053863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af542d2-61b7-4e9f-8707-b7f72e910dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:44.122868 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1af542d2-61b7-4e9f-8707-b7f72e910dd0 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:44.124125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1af542d2-61b7-4e9f-8707-b7f72e910dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:44.142789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:44.229081 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 10:37:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:37:44.268476 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0ab9b-4453-457c-8c50-11f2b5998f2d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:44.270106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e2d0ab9b-4453-457c-8c50-11f2b5998f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:44.388297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0ab9b-4453-457c-8c50-11f2b5998f2d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:44.465205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:44.473092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d02556-529a-4e92-9a3c-6886eb7f85c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:44.473783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d02556-529a-4e92-9a3c-6886eb7f85c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:44.474050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d02556-529a-4e92-9a3c-6886eb7f85c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:44.474321 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d02556-529a-4e92-9a3c-6886eb7f85c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:44.475983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-41d02556-529a-4e92-9a3c-6886eb7f85c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:44.730006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d02556-529a-4e92-9a3c-6886eb7f85c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:44.731393 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d02556-529a-4e92-9a3c-6886eb7f85c4 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:44.752956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 726/1467] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:46.834764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247053f-2915-446e-94b9-f6a2ede60d71 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:46.835382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247053f-2915-446e-94b9-f6a2ede60d71 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:46.835382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247053f-2915-446e-94b9-f6a2ede60d71 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:46.836692 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 10:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:37:46.842060 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8603e0-a57f-4a1d-8d81-b91d4b443998 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:46.843291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8603e0-a57f-4a1d-8d81-b91d4b443998 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:46.843291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8603e0-a57f-4a1d-8d81-b91d4b443998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:46.910924 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2a8603e0-a57f-4a1d-8d81-b91d4b443998 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:37:46.912038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2a8603e0-a57f-4a1d-8d81-b91d4b443998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:46.961234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:47.014719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 10:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:37:47.052081 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247053f-2915-446e-94b9-f6a2ede60d71 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:37:47.053700 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2247053f-2915-446e-94b9-f6a2ede60d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:47.163522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247053f-2915-446e-94b9-f6a2ede60d71 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:37:47.215893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:47.247332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a103b2-2fa0-44c2-916d-63f7264a4d3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:37:47.247989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a103b2-2fa0-44c2-916d-63f7264a4d3c service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:37:47.248268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a103b2-2fa0-44c2-916d-63f7264a4d3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:37:47.249078 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a103b2-2fa0-44c2-916d-63f7264a4d3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:37:47.250809 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-97a103b2-2fa0-44c2-916d-63f7264a4d3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:37:47.384447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a103b2-2fa0-44c2-916d-63f7264a4d3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:37:47.386052 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a103b2-2fa0-44c2-916d-63f7264a4d3c service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:37:47.405806 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 743/1470] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:37:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:37:57.012213 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=22 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:37:57.403281 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=66 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:38:07.062977 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9142de7-05f3-4cc8-b7bc-c22503a9a154 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:07.063563 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9142de7-05f3-4cc8-b7bc-c22503a9a154 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:07.063563 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9142de7-05f3-4cc8-b7bc-c22503a9a154 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:07.064874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 728/1471] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:38:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:38:07.070335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7dd5b-afaa-4a29-95a6-91faaa3c69e6 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:07.070783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7dd5b-afaa-4a29-95a6-91faaa3c69e6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:07.070905 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7dd5b-afaa-4a29-95a6-91faaa3c69e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:07.140186 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-47e7dd5b-afaa-4a29-95a6-91faaa3c69e6 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:38:07.141420 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-47e7dd5b-afaa-4a29-95a6-91faaa3c69e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:07.238082 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 744/1472] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:38:07.324827 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f016c-4733-4210-b264-13b48ac3b8ce None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:07.325113 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f016c-4733-4210-b264-13b48ac3b8ce None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:07.325405 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f016c-4733-4210-b264-13b48ac3b8ce None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:07.326796 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 729/1473] 10.222.0.47 () {54 vars in 966 bytes} [Wed Aug 3 10:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:38:07.334289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27d89a-76aa-435c-a499-a1de320bf0f5 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:07.334623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27d89a-76aa-435c-a499-a1de320bf0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:07.334939 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27d89a-76aa-435c-a499-a1de320bf0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:07.413441 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7d27d89a-76aa-435c-a499-a1de320bf0f5 None None] MFA Rules not processed for user `300fb5e1c5c546808b935871673e3d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:38:07.414709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7d27d89a-76aa-435c-a499-a1de320bf0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:07.436777 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:38:07.494634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 745/1474] 10.222.0.47 () {58 vars in 1063 bytes} [Wed Aug 3 10:38:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:38:07.502357 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f016c-4733-4210-b264-13b48ac3b8ce None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:38:07.503902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e75f016c-4733-4210-b264-13b48ac3b8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:07.613104 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f016c-4733-4210-b264-13b48ac3b8ce None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:38:07.688354 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87b2e0-fac4-430a-9542-f47fe392a682 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '300fb5e1c5c546808b935871673e3d0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:38:07.689118 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87b2e0-fac4-430a-9542-f47fe392a682 service glance] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:07.689403 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87b2e0-fac4-430a-9542-f47fe392a682 service glance] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:07.689706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87b2e0-fac4-430a-9542-f47fe392a682 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:07.691348 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2b87b2e0-fac4-430a-9542-f47fe392a682 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:07.748676 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:38:07.803689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87b2e0-fac4-430a-9542-f47fe392a682 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:38:07.805121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87b2e0-fac4-430a-9542-f47fe392a682 service glance] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:38:07.824126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 730/1475] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:38:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:38:09.910641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83da7-e103-451a-a8a4-09a1b05ba8fd None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:09.911216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83da7-e103-451a-a8a4-09a1b05ba8fd None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:09.911216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83da7-e103-451a-a8a4-09a1b05ba8fd None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:09.912600 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 746/1476] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:38:09.917966 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc8c3f-15cf-47c2-8a2b-13b8d979ec08 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:09.918419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc8c3f-15cf-47c2-8a2b-13b8d979ec08 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:09.918543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc8c3f-15cf-47c2-8a2b-13b8d979ec08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:09.990234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-19dc8c3f-15cf-47c2-8a2b-13b8d979ec08 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:38:09.991464 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-19dc8c3f-15cf-47c2-8a2b-13b8d979ec08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:10.088352 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 731/1477] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:38:10.111796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca83da7-e103-451a-a8a4-09a1b05ba8fd None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:38:10.113322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1ca83da7-e103-451a-a8a4-09a1b05ba8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:10.248874 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca83da7-e103-451a-a8a4-09a1b05ba8fd None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:38:10.298208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:38:10.343602 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c1992-d4d4-4f48-9dad-a1f11d5ba8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:38:10.344224 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c1992-d4d4-4f48-9dad-a1f11d5ba8fa admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:10.344508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c1992-d4d4-4f48-9dad-a1f11d5ba8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:10.344783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c1992-d4d4-4f48-9dad-a1f11d5ba8fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:10.345589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c1992-d4d4-4f48-9dad-a1f11d5ba8fa admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:38:10.347327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: WARNING py.warnings [None req-8a1c1992-d4d4-4f48-9dad-a1f11d5ba8fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 10:38:10.347327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: warnings.warn(msg) Aug 03 10:38:10.347327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: Aug 03 10:38:10.348739 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c1992-d4d4-4f48-9dad-a1f11d5ba8fa admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:38:10.367698 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 747/1478] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 3 10:38:10 2022] GET /identity/v3/projects => generated 2553 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:38:12.400512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90b422-b749-4bc6-8631-5bd47d19e14c None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:12.401413 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90b422-b749-4bc6-8631-5bd47d19e14c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:12.401413 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90b422-b749-4bc6-8631-5bd47d19e14c None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:12.402495 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 732/1479] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:38:12.408497 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38c0ad-c44d-4f11-99fb-6a2a4a8b4f08 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:12.409041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38c0ad-c44d-4f11-99fb-6a2a4a8b4f08 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:12.409194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38c0ad-c44d-4f11-99fb-6a2a4a8b4f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:12.475870 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3b38c0ad-c44d-4f11-99fb-6a2a4a8b4f08 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:38:12.477098 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3b38c0ad-c44d-4f11-99fb-6a2a4a8b4f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:12.572546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 748/1480] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:38:12.615405 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90b422-b749-4bc6-8631-5bd47d19e14c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:38:12.616961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4a90b422-b749-4bc6-8631-5bd47d19e14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:12.627214 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:38:12.764160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90b422-b749-4bc6-8631-5bd47d19e14c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:38:12.845148 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:38:12.878485 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d5ab1-ea17-4226-9385-d71999b0bc36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:38:12.879138 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d5ab1-ea17-4226-9385-d71999b0bc36 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:12.879416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d5ab1-ea17-4226-9385-d71999b0bc36 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:12.879689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d5ab1-ea17-4226-9385-d71999b0bc36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:12.881312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ca9d5ab1-ea17-4226-9385-d71999b0bc36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:12.968621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d5ab1-ea17-4226-9385-d71999b0bc36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:38:12.970246 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d5ab1-ea17-4226-9385-d71999b0bc36 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:38:12.988784 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 733/1481] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:38:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:38:15.177246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1bf18-542e-4bc8-9f06-e3b225c5039b None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:15.177881 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1bf18-542e-4bc8-9f06-e3b225c5039b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:15.177881 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1bf18-542e-4bc8-9f06-e3b225c5039b None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:15.179235 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 749/1482] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:38:15.184361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a356224-0f84-4ee9-b015-a8a011a00be7 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:15.184830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a356224-0f84-4ee9-b015-a8a011a00be7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:15.184940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a356224-0f84-4ee9-b015-a8a011a00be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:15.253411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3a356224-0f84-4ee9-b015-a8a011a00be7 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:38:15.254644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3a356224-0f84-4ee9-b015-a8a011a00be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:15.301276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:38:15.353204 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 734/1483] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:38:15.388824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1bf18-542e-4bc8-9f06-e3b225c5039b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:38:15.390941 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-92a1bf18-542e-4bc8-9f06-e3b225c5039b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:15.415554 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:38:15.505328 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1bf18-542e-4bc8-9f06-e3b225c5039b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:38:15.578528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01649066-d10c-4097-8378-725d120ac867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:38:15.579149 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01649066-d10c-4097-8378-725d120ac867 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:15.579430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01649066-d10c-4097-8378-725d120ac867 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:15.579703 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01649066-d10c-4097-8378-725d120ac867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:15.581351 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-01649066-d10c-4097-8378-725d120ac867 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:15.660724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:38:15.697306 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01649066-d10c-4097-8378-725d120ac867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:38:15.698778 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01649066-d10c-4097-8378-725d120ac867 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:38:15.718874 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 750/1484] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:38:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:38:17.817401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4379d-4420-42e1-93bb-6a51d22925dd None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:17.818012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4379d-4420-42e1-93bb-6a51d22925dd None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:17.818012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4379d-4420-42e1-93bb-6a51d22925dd None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:17.819518 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 735/1485] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:38:17.824872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88e897-db4d-469d-9ca8-f047f8a469c1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:17.825345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88e897-db4d-469d-9ca8-f047f8a469c1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:17.825530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88e897-db4d-469d-9ca8-f047f8a469c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:17.898503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ae88e897-db4d-469d-9ca8-f047f8a469c1 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:38:17.899726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ae88e897-db4d-469d-9ca8-f047f8a469c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:17.999759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 751/1486] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:38:18.038263 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4379d-4420-42e1-93bb-6a51d22925dd None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:38:18.039821 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f4b4379d-4420-42e1-93bb-6a51d22925dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:18.142342 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4379d-4420-42e1-93bb-6a51d22925dd None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:38:18.211869 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:38:18.213633 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8762290-1f93-4c22-b929-78d2e49e3840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:38:18.214269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8762290-1f93-4c22-b929-78d2e49e3840 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:18.214731 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8762290-1f93-4c22-b929-78d2e49e3840 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:18.215050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8762290-1f93-4c22-b929-78d2e49e3840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:18.217083 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f8762290-1f93-4c22-b929-78d2e49e3840 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:18.327222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8762290-1f93-4c22-b929-78d2e49e3840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:38:18.328557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8762290-1f93-4c22-b929-78d2e49e3840 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:38:18.346230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 736/1487] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:38:20.499694 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd6cd1-baed-4a3b-a565-f5b2a3dd8170 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:20.500274 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd6cd1-baed-4a3b-a565-f5b2a3dd8170 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:20.500274 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd6cd1-baed-4a3b-a565-f5b2a3dd8170 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:20.501679 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 752/1488] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:38:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:38:20.507061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b49872-227e-4d09-8095-fff70c7add43 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:20.507500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b49872-227e-4d09-8095-fff70c7add43 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:20.507609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b49872-227e-4d09-8095-fff70c7add43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:20.573598 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d6b49872-227e-4d09-8095-fff70c7add43 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:38:20.574822 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d6b49872-227e-4d09-8095-fff70c7add43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:20.629187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:38:20.674894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 737/1489] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:38:20.715648 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd6cd1-baed-4a3b-a565-f5b2a3dd8170 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:38:20.717231 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c0dd6cd1-baed-4a3b-a565-f5b2a3dd8170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:20.754235 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:38:20.825010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd6cd1-baed-4a3b-a565-f5b2a3dd8170 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:38:20.901678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8c0f6-7418-4f11-af81-8b303698aa81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:38:20.902304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8c0f6-7418-4f11-af81-8b303698aa81 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:20.902585 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8c0f6-7418-4f11-af81-8b303698aa81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:20.902856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8c0f6-7418-4f11-af81-8b303698aa81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:20.904515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-23e8c0f6-7418-4f11-af81-8b303698aa81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:20.986493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:38:21.015211 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8c0f6-7418-4f11-af81-8b303698aa81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:38:21.016582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8c0f6-7418-4f11-af81-8b303698aa81 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:38:21.034852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 753/1490] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:38:23.258391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57133ecf-128e-4b56-95bb-ea305b850e43 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:23.258858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57133ecf-128e-4b56-95bb-ea305b850e43 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:23.258987 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57133ecf-128e-4b56-95bb-ea305b850e43 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:23.260450 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 738/1491] 10.222.0.47 () {54 vars in 926 bytes} [Wed Aug 3 10:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:38:23.266487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97080df2-1c85-44a9-b4ea-f576bac37d02 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:23.266928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97080df2-1c85-44a9-b4ea-f576bac37d02 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:23.267026 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97080df2-1c85-44a9-b4ea-f576bac37d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:23.332721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-97080df2-1c85-44a9-b4ea-f576bac37d02 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:38:23.333978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-97080df2-1c85-44a9-b4ea-f576bac37d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:23.428988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 754/1492] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Aug 3 10:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:38:23.469222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57133ecf-128e-4b56-95bb-ea305b850e43 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:38:23.470792 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-57133ecf-128e-4b56-95bb-ea305b850e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:23.576495 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57133ecf-128e-4b56-95bb-ea305b850e43 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:38:23.650610 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99649a-c0d2-4200-9a4b-c44b82445d9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:38:23.651240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99649a-c0d2-4200-9a4b-c44b82445d9a service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:38:23.651524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99649a-c0d2-4200-9a4b-c44b82445d9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:38:23.651801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99649a-c0d2-4200-9a4b-c44b82445d9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:38:23.653433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8c99649a-c0d2-4200-9a4b-c44b82445d9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:38:23.663135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:38:23.774377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99649a-c0d2-4200-9a4b-c44b82445d9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:38:23.775851 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99649a-c0d2-4200-9a4b-c44b82445d9a service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:38:23.793313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 739/1493] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:38:33.427152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=82 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:38:33.791086 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=50 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:39:51.990762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d941bc2-6877-46e2-8846-71fe4109c4e1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:51.991343 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d941bc2-6877-46e2-8846-71fe4109c4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:51.991343 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d941bc2-6877-46e2-8846-71fe4109c4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:52.066675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5d941bc2-6877-46e2-8846-71fe4109c4e1 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:39:52.067889 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5d941bc2-6877-46e2-8846-71fe4109c4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:52.162681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 755/1494] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:39:52.170717 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40ad28-9b92-4f0a-8291-af726a20c40e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:52.171213 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40ad28-9b92-4f0a-8291-af726a20c40e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:52.171316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40ad28-9b92-4f0a-8291-af726a20c40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:52.246676 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-bd40ad28-9b92-4f0a-8291-af726a20c40e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:39:52.247816 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bd40ad28-9b92-4f0a-8291-af726a20c40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:52.343284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 740/1495] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:39:52.348071 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d941bc2-6877-46e2-8846-71fe4109c4e1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:39:52.349422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5d941bc2-6877-46e2-8846-71fe4109c4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:52.361263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d941bc2-6877-46e2-8846-71fe4109c4e1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:39:52.423621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:39:52.581424 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42b79f-5dac-4bc6-9df6-8f10d3902890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:39:52.581992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42b79f-5dac-4bc6-9df6-8f10d3902890 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:52.582275 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42b79f-5dac-4bc6-9df6-8f10d3902890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:52.582550 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42b79f-5dac-4bc6-9df6-8f10d3902890 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:52.583247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42b79f-5dac-4bc6-9df6-8f10d3902890 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:39:52.583553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42b79f-5dac-4bc6-9df6-8f10d3902890 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:39:52.584556 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: WARNING py.warnings [None req-1f42b79f-5dac-4bc6-9df6-8f10d3902890 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 10:39:52.584556 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: warnings.warn(msg) Aug 03 10:39:52.584556 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: Aug 03 10:39:52.585441 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42b79f-5dac-4bc6-9df6-8f10d3902890 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:39:52.597151 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 756/1496] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:39:52.605738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40ad28-9b92-4f0a-8291-af726a20c40e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:39:52.614850 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40ad28-9b92-4f0a-8291-af726a20c40e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:39:52.675331 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:39:52.710461 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95189a-8706-4a0c-a5b5-0d071af406dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:39:52.711089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95189a-8706-4a0c-a5b5-0d071af406dd admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:52.711370 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95189a-8706-4a0c-a5b5-0d071af406dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:52.711644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95189a-8706-4a0c-a5b5-0d071af406dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:52.712319 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95189a-8706-4a0c-a5b5-0d071af406dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:39:52.714088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: WARNING py.warnings [None req-2a95189a-8706-4a0c-a5b5-0d071af406dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 10:39:52.714088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: warnings.warn(msg) Aug 03 10:39:52.714088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: Aug 03 10:39:52.715281 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95189a-8706-4a0c-a5b5-0d071af406dd admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:39:52.771246 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 741/1497] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:39:52 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:39:52.775560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42b79f-5dac-4bc6-9df6-8f10d3902890 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:39:52.777068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1f42b79f-5dac-4bc6-9df6-8f10d3902890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:52.809950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42b79f-5dac-4bc6-9df6-8f10d3902890 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:39:52.903549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98e7c3-5ceb-462c-aa47-3fc0a4a0828c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:39:52.904086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98e7c3-5ceb-462c-aa47-3fc0a4a0828c admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:52.904363 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98e7c3-5ceb-462c-aa47-3fc0a4a0828c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:52.904635 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98e7c3-5ceb-462c-aa47-3fc0a4a0828c admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:52.905297 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98e7c3-5ceb-462c-aa47-3fc0a4a0828c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:39:52.906326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: WARNING py.warnings [None req-3b98e7c3-5ceb-462c-aa47-3fc0a4a0828c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 10:39:52.906326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: warnings.warn(msg) Aug 03 10:39:52.906326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: Aug 03 10:39:52.907512 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98e7c3-5ceb-462c-aa47-3fc0a4a0828c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:39:52.949109 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 757/1498] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:39:52 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:39:52.953230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95189a-8706-4a0c-a5b5-0d071af406dd admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:39:52.954800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2a95189a-8706-4a0c-a5b5-0d071af406dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:53.015778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95189a-8706-4a0c-a5b5-0d071af406dd admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:39:53.095716 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:39:53.107396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6a27c-3eca-495b-aed2-95690423f6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:39:53.108638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6a27c-3eca-495b-aed2-95690423f6af admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:53.108922 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6a27c-3eca-495b-aed2-95690423f6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:53.109190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6a27c-3eca-495b-aed2-95690423f6af admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:53.109914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6a27c-3eca-495b-aed2-95690423f6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:39:53.110924 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: WARNING py.warnings [None req-3ac6a27c-3eca-495b-aed2-95690423f6af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 10:39:53.110924 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: warnings.warn(msg) Aug 03 10:39:53.110924 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: Aug 03 10:39:53.111807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6a27c-3eca-495b-aed2-95690423f6af admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:39:53.124545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 742/1499] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:39:53.128596 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98e7c3-5ceb-462c-aa47-3fc0a4a0828c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:39:53.130201 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3b98e7c3-5ceb-462c-aa47-3fc0a4a0828c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:53.141594 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98e7c3-5ceb-462c-aa47-3fc0a4a0828c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:39:53.167336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:39:53.237403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994434f-b9ce-4697-8472-d1e7205561f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:39:53.238052 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994434f-b9ce-4697-8472-d1e7205561f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:53.238329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994434f-b9ce-4697-8472-d1e7205561f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:53.238598 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994434f-b9ce-4697-8472-d1e7205561f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:53.239290 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994434f-b9ce-4697-8472-d1e7205561f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:39:53.240292 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: WARNING py.warnings [None req-e994434f-b9ce-4697-8472-d1e7205561f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 10:39:53.240292 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: warnings.warn(msg) Aug 03 10:39:53.240292 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: Aug 03 10:39:53.241164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994434f-b9ce-4697-8472-d1e7205561f9 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:39:53.253806 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 758/1500] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:39:53.258234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6a27c-3eca-495b-aed2-95690423f6af admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:39:53.259758 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3ac6a27c-3eca-495b-aed2-95690423f6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:53.293532 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6a27c-3eca-495b-aed2-95690423f6af admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:39:53.385899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631485da-3af2-4e27-8742-2ea5089468b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:39:53.386568 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631485da-3af2-4e27-8742-2ea5089468b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:53.386848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631485da-3af2-4e27-8742-2ea5089468b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:53.387121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631485da-3af2-4e27-8742-2ea5089468b7 admin admin] PATH_INFO: `/v3/projects/74124589adb04351aa983833f89e302d/users/b9fbdf67b17a4597976f221c2c56b267/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:53.409424 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631485da-3af2-4e27-8742-2ea5089468b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74124589adb04351aa983833f89e302d, user_id=b9fbdf67b17a4597976f221c2c56b267, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:39:53.410486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: WARNING py.warnings [None req-631485da-3af2-4e27-8742-2ea5089468b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 10:39:53.410486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: warnings.warn(msg) Aug 03 10:39:53.410486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: Aug 03 10:39:53.412407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631485da-3af2-4e27-8742-2ea5089468b7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:39:53.436866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 743/1501] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:39:53 2022] PUT /identity/v3/projects/74124589adb04351aa983833f89e302d/users/b9fbdf67b17a4597976f221c2c56b267/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:39:53.449132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f660f9-49aa-4872-94e9-06ddfa39c216 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:53.449511 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f660f9-49aa-4872-94e9-06ddfa39c216 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:53.449801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f660f9-49aa-4872-94e9-06ddfa39c216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:53.505747 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-16f660f9-49aa-4872-94e9-06ddfa39c216 None None] MFA Rules not processed for user `b9fbdf67b17a4597976f221c2c56b267`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:39:53.506944 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-16f660f9-49aa-4872-94e9-06ddfa39c216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:53.583355 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 759/1502] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:39:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:39:53.596862 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8105d-021b-4fb6-b7fd-970d8365cc19 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:53.597155 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8105d-021b-4fb6-b7fd-970d8365cc19 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:53.597493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8105d-021b-4fb6-b7fd-970d8365cc19 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:53.598872 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 744/1503] 10.222.0.47 () {54 vars in 964 bytes} [Wed Aug 3 10:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:39:53.604255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7ffde-bbfd-4a30-81fd-b06febbf53a8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:53.604539 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7ffde-bbfd-4a30-81fd-b06febbf53a8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:53.604819 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7ffde-bbfd-4a30-81fd-b06febbf53a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:53.634701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:39:53.680619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7be7ffde-bbfd-4a30-81fd-b06febbf53a8 None None] MFA Rules not processed for user `3b5f547a388c4f64aeb95a7435818741`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:39:53.681887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7be7ffde-bbfd-4a30-81fd-b06febbf53a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:53.756369 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 760/1504] 10.222.0.47 () {58 vars in 1061 bytes} [Wed Aug 3 10:39:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:39:53.764583 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8105d-021b-4fb6-b7fd-970d8365cc19 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:39:53.766125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-70e8105d-021b-4fb6-b7fd-970d8365cc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:53.816014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:39:53.870850 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8105d-021b-4fb6-b7fd-970d8365cc19 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:39:53.943045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e80d4-2319-4357-9e25-b7926ac92cce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:39:53.943661 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e80d4-2319-4357-9e25-b7926ac92cce service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:53.943936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e80d4-2319-4357-9e25-b7926ac92cce service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:53.944206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e80d4-2319-4357-9e25-b7926ac92cce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:53.945833 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0f1e80d4-2319-4357-9e25-b7926ac92cce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:54.043988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e80d4-2319-4357-9e25-b7926ac92cce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:39:54.045319 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e80d4-2319-4357-9e25-b7926ac92cce service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:39:54.048758 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:39:54.064023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 745/1505] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:39:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:39:54.109244 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d895ce-b065-4b10-9b04-5dd14ab5813d None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:54.109634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d895ce-b065-4b10-9b04-5dd14ab5813d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:54.109913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d895ce-b065-4b10-9b04-5dd14ab5813d None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:54.111186 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 761/1506] 10.222.0.47 () {52 vars in 830 bytes} [Wed Aug 3 10:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:39:54.114792 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e80d4-2319-4357-9e25-b7926ac92cce service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:39:54.116055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0f1e80d4-2319-4357-9e25-b7926ac92cce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:54.212463 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e80d4-2319-4357-9e25-b7926ac92cce service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:39:54.304538 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b1a7f-3cad-456f-b836-dbb69c5c69fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:39:54.305197 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b1a7f-3cad-456f-b836-dbb69c5c69fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:54.305360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b1a7f-3cad-456f-b836-dbb69c5c69fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:54.305667 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b1a7f-3cad-456f-b836-dbb69c5c69fb admin admin] PATH_INFO: `/v3/users/b9fbdf67b17a4597976f221c2c56b267` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:54.318130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b1a7f-3cad-456f-b836-dbb69c5c69fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9fbdf67b17a4597976f221c2c56b267)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:39:54.319155 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: WARNING py.warnings [None req-626b1a7f-3cad-456f-b836-dbb69c5c69fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 10:39:54.319155 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: warnings.warn(msg) Aug 03 10:39:54.319155 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: Aug 03 10:39:54.320365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b1a7f-3cad-456f-b836-dbb69c5c69fb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:39:54.330603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:39:54.389232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-626b1a7f-3cad-456f-b836-dbb69c5c69fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9fbdf67b17a4597976f221c2c56b267'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:39:54.404844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-626b1a7f-3cad-456f-b836-dbb69c5c69fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9fbdf67b17a4597976f221c2c56b267'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:39:54.413860 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-626b1a7f-3cad-456f-b836-dbb69c5c69fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9fbdf67b17a4597976f221c2c56b267'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:39:54.414386 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-626b1a7f-3cad-456f-b836-dbb69c5c69fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9fbdf67b17a4597976f221c2c56b267'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:39:54.414870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-626b1a7f-3cad-456f-b836-dbb69c5c69fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9fbdf67b17a4597976f221c2c56b267'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:39:54.443874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 746/1507] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:39:54 2022] DELETE /identity/v3/users/b9fbdf67b17a4597976f221c2c56b267 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:39:54.451805 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9fd5e-70d0-4977-b5b8-d3948f431db5 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:54.452312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9fd5e-70d0-4977-b5b8-d3948f431db5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:54.452433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9fd5e-70d0-4977-b5b8-d3948f431db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:54.508664 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c6f9fd5e-70d0-4977-b5b8-d3948f431db5 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:39:54.509925 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c6f9fd5e-70d0-4977-b5b8-d3948f431db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:54.588803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:39:54.601699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 762/1508] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:39:54.611803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b1a7f-3cad-456f-b836-dbb69c5c69fb admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:39:54.613038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-626b1a7f-3cad-456f-b836-dbb69c5c69fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:54.724345 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b1a7f-3cad-456f-b836-dbb69c5c69fb admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:39:54.785374 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:39:54.807935 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29cfd4-e66c-4477-882f-09eba5dc669b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:39:54.808569 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29cfd4-e66c-4477-882f-09eba5dc669b service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:54.808880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29cfd4-e66c-4477-882f-09eba5dc669b service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:54.809156 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29cfd4-e66c-4477-882f-09eba5dc669b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:54.810855 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3d29cfd4-e66c-4477-882f-09eba5dc669b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:54.916829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29cfd4-e66c-4477-882f-09eba5dc669b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:39:54.918186 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29cfd4-e66c-4477-882f-09eba5dc669b service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:39:54.936695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 747/1509] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:39:55.273911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9fd5e-70d0-4977-b5b8-d3948f431db5 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:39:55.275186 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c6f9fd5e-70d0-4977-b5b8-d3948f431db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:39:55.287265 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9fd5e-70d0-4977-b5b8-d3948f431db5 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:39:55.385820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:39:55.386466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:39:55.386742 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:39:55.387021 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870 admin admin] PATH_INFO: `/v3/projects/74124589adb04351aa983833f89e302d` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:39:55.397431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74124589adb04351aa983833f89e302d)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:39:55.398481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: WARNING py.warnings [None req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 10:39:55.398481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: warnings.warn(msg) Aug 03 10:39:55.398481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: Aug 03 10:39:55.399677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:39:55.459931 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74124589adb04351aa983833f89e302d', 'request_id': 'req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:39:55.470300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74124589adb04351aa983833f89e302d', 'request_id': 'req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:39:55.484722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74124589adb04351aa983833f89e302d', 'request_id': 'req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:39:55.506731 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870 admin admin] The token cache is being invalidate because project 74124589adb04351aa983833f89e302d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:39:55.507022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:39:55.507555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c34c9ee9-af0a-421d-8cdd-0ea91dcbc870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:39:55.508884 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 763/1510] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:39:55 2022] DELETE /identity/v3/projects/74124589adb04351aa983833f89e302d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:40:04.934539 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=62 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:40:05.504718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:41:32.074926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f40a69-036b-4c1f-a60b-04048f01f70e None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:32.075544 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f40a69-036b-4c1f-a60b-04048f01f70e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:32.075544 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f40a69-036b-4c1f-a60b-04048f01f70e None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:32.076887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 10:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:41:32.082965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bd781-c9e8-46d2-9bb2-1a84a5e1e5fd None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:32.083438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bd781-c9e8-46d2-9bb2-1a84a5e1e5fd None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:32.083545 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bd781-c9e8-46d2-9bb2-1a84a5e1e5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:32.168033 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1c4bd781-c9e8-46d2-9bb2-1a84a5e1e5fd None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:41:32.169260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1c4bd781-c9e8-46d2-9bb2-1a84a5e1e5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:41:32.285683 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 10:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:41:32.452433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f40a69-036b-4c1f-a60b-04048f01f70e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:41:32.454578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f6f40a69-036b-4c1f-a60b-04048f01f70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:41:32.574593 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f40a69-036b-4c1f-a60b-04048f01f70e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:41:32.580846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5356a40-7ea7-4a8e-bcba-9d8c7c34f9aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:41:32.581494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5356a40-7ea7-4a8e-bcba-9d8c7c34f9aa service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:32.581796 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5356a40-7ea7-4a8e-bcba-9d8c7c34f9aa service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:32.582069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5356a40-7ea7-4a8e-bcba-9d8c7c34f9aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:32.590732 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5356a40-7ea7-4a8e-bcba-9d8c7c34f9aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:41:32.592333 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5356a40-7ea7-4a8e-bcba-9d8c7c34f9aa service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:41:32.602650 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 749/1513] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:41:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:41:33.457916 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725bd12-99d8-436d-8090-e5a016c2b3c7 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:33.458531 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725bd12-99d8-436d-8090-e5a016c2b3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:33.458753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725bd12-99d8-436d-8090-e5a016c2b3c7 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:33.460928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 10:41:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:41:33.466232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71d44a-09d0-460d-a65a-42db6b376a9e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:33.466716 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71d44a-09d0-460d-a65a-42db6b376a9e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:33.466828 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71d44a-09d0-460d-a65a-42db6b376a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:33.488164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-9e71d44a-09d0-460d-a65a-42db6b376a9e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:41:33.489359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9e71d44a-09d0-460d-a65a-42db6b376a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:41:33.497800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 10:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:41:33.651639 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725bd12-99d8-436d-8090-e5a016c2b3c7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:41:33.666494 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725bd12-99d8-436d-8090-e5a016c2b3c7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:41:33.672005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9216ed-ca0f-4af2-8c12-344820cb880f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:41:33.672622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9216ed-ca0f-4af2-8c12-344820cb880f service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:33.673510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9216ed-ca0f-4af2-8c12-344820cb880f service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:33.674404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9216ed-ca0f-4af2-8c12-344820cb880f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:33.684454 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9216ed-ca0f-4af2-8c12-344820cb880f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:41:33.686522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9216ed-ca0f-4af2-8c12-344820cb880f service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:41:33.697877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 766/1516] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:41:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:41:34.261364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db35c75-829c-4547-bef7-eecffbbb41f8 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:34.261693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db35c75-829c-4547-bef7-eecffbbb41f8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:34.262059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db35c75-829c-4547-bef7-eecffbbb41f8 None None] PATH_INFO: `/v3` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:34.263306 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 751/1517] 10.222.0.61 () {54 vars in 925 bytes} [Wed Aug 3 10:41:34 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:41:34.270455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ead22d-efc5-4acc-bf1f-c648face2a36 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:34.270750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ead22d-efc5-4acc-bf1f-c648face2a36 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:34.271028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ead22d-efc5-4acc-bf1f-c648face2a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:34.333398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-39ead22d-efc5-4acc-bf1f-c648face2a36 None None] MFA Rules not processed for user `f2c5536eddd346d692822a2f3befb48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:41:34.334698 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-39ead22d-efc5-4acc-bf1f-c648face2a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:41:34.354377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:41:34.418489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 767/1518] 10.222.0.61 () {58 vars in 1015 bytes} [Wed Aug 3 10:41:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:41:38.184857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265093a2-738c-4f8d-bf81-b65431a61d7f None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:38.185560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265093a2-738c-4f8d-bf81-b65431a61d7f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:38.185560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265093a2-738c-4f8d-bf81-b65431a61d7f None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:38.186851 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 752/1519] 10.222.0.47 () {54 vars in 961 bytes} [Wed Aug 3 10:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:41:38.192283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f89d8-74d0-4c69-8944-f410e52d6c0f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:38.192727 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f89d8-74d0-4c69-8944-f410e52d6c0f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:38.192826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f89d8-74d0-4c69-8944-f410e52d6c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:38.212354 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2d4f89d8-74d0-4c69-8944-f410e52d6c0f None None] MFA Rules not processed for user `f2c5536eddd346d692822a2f3befb48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:41:38.213578 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2d4f89d8-74d0-4c69-8944-f410e52d6c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:41:38.222120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 768/1520] 10.222.0.47 () {58 vars in 1058 bytes} [Wed Aug 3 10:41:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:41:38.230394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265093a2-738c-4f8d-bf81-b65431a61d7f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:41:38.243368 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265093a2-738c-4f8d-bf81-b65431a61d7f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:41:38.248741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef387209-54aa-4a2e-a1c7-985c54e62965 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2c5536eddd346d692822a2f3befb48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:41:38.249344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef387209-54aa-4a2e-a1c7-985c54e62965 service placement] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:38.249657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef387209-54aa-4a2e-a1c7-985c54e62965 service placement] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:38.249938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef387209-54aa-4a2e-a1c7-985c54e62965 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:38.258725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef387209-54aa-4a2e-a1c7-985c54e62965 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:41:38.260663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef387209-54aa-4a2e-a1c7-985c54e62965 service placement] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:41:38.270930 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 753/1521] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:41:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:41:44.922472 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f8da1-5879-4957-89c0-923011af4d96 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:44.923014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f8da1-5879-4957-89c0-923011af4d96 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:44.923014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f8da1-5879-4957-89c0-923011af4d96 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:44.925032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 10:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:41:44.930281 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569eb069-6777-40c9-bf4a-057118e2ddc4 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:44.930786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569eb069-6777-40c9-bf4a-057118e2ddc4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:44.930934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569eb069-6777-40c9-bf4a-057118e2ddc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:44.950009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-569eb069-6777-40c9-bf4a-057118e2ddc4 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:41:44.951249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-569eb069-6777-40c9-bf4a-057118e2ddc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:41:44.959237 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 10:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:41:45.115969 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f8da1-5879-4957-89c0-923011af4d96 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:41:45.130401 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f8da1-5879-4957-89c0-923011af4d96 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:41:45.135787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21be977-8c2a-4c1c-b948-8bed2f6a2e5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:41:45.136387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21be977-8c2a-4c1c-b948-8bed2f6a2e5a service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:45.136666 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21be977-8c2a-4c1c-b948-8bed2f6a2e5a service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:45.136935 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21be977-8c2a-4c1c-b948-8bed2f6a2e5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:45.145040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21be977-8c2a-4c1c-b948-8bed2f6a2e5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:41:45.146741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21be977-8c2a-4c1c-b948-8bed2f6a2e5a service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:41:45.155969 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 770/1524] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:41:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:41:45.980360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90327070-dc4f-406c-bbd7-c94ef96127b9 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:45.980898 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90327070-dc4f-406c-bbd7-c94ef96127b9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:45.980898 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90327070-dc4f-406c-bbd7-c94ef96127b9 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:45.982313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 10:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:41:45.987228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafaa8b-4cdb-40aa-ade5-0fd674542f6f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:45.987678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafaa8b-4cdb-40aa-ade5-0fd674542f6f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:45.987789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafaa8b-4cdb-40aa-ade5-0fd674542f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:46.007443 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6dafaa8b-4cdb-40aa-ade5-0fd674542f6f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:41:46.008637 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6dafaa8b-4cdb-40aa-ade5-0fd674542f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:41:46.016561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 10:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:41:46.169565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90327070-dc4f-406c-bbd7-c94ef96127b9 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:41:46.183827 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90327070-dc4f-406c-bbd7-c94ef96127b9 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:41:46.189220 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27ace0-50bf-4acd-aae5-76c902bd3739 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:41:46.189869 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27ace0-50bf-4acd-aae5-76c902bd3739 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:46.190157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27ace0-50bf-4acd-aae5-76c902bd3739 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:46.190434 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27ace0-50bf-4acd-aae5-76c902bd3739 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:46.198484 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27ace0-50bf-4acd-aae5-76c902bd3739 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:41:46.200089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27ace0-50bf-4acd-aae5-76c902bd3739 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:41:46.210033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 756/1527] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:41:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:41:49.966425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34337ae3-58e4-4288-a7ba-f592302319f8 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:49.967012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34337ae3-58e4-4288-a7ba-f592302319f8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:49.967012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34337ae3-58e4-4288-a7ba-f592302319f8 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:49.968335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 10:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:41:49.973532 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b533cf8-2534-4a85-b55f-b3e436248025 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:49.973955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b533cf8-2534-4a85-b55f-b3e436248025 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:49.974098 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b533cf8-2534-4a85-b55f-b3e436248025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:49.995895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7b533cf8-2534-4a85-b55f-b3e436248025 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:41:49.997098 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7b533cf8-2534-4a85-b55f-b3e436248025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:41:50.005540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 10:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:41:50.081699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34337ae3-58e4-4288-a7ba-f592302319f8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:41:50.083426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-34337ae3-58e4-4288-a7ba-f592302319f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:41:50.161445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34337ae3-58e4-4288-a7ba-f592302319f8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:41:50.165094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa306aba-f2ef-4649-aec9-aa615b11fe8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:41:50.165736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa306aba-f2ef-4649-aec9-aa615b11fe8f service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:50.166023 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa306aba-f2ef-4649-aec9-aa615b11fe8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:50.166301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa306aba-f2ef-4649-aec9-aa615b11fe8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:50.176195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa306aba-f2ef-4649-aec9-aa615b11fe8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:41:50.177649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa306aba-f2ef-4649-aec9-aa615b11fe8f service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:41:50.187196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 773/1530] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:41:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:41:52.152708 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54d76e-045c-431b-9703-a7d2afa511f5 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:52.153280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54d76e-045c-431b-9703-a7d2afa511f5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:52.153280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54d76e-045c-431b-9703-a7d2afa511f5 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:52.154699 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 10:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:41:52.160577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64f9d7-3fe0-4874-9236-fdc61b10d692 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:52.161005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64f9d7-3fe0-4874-9236-fdc61b10d692 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:52.161125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64f9d7-3fe0-4874-9236-fdc61b10d692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:52.180180 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ad64f9d7-3fe0-4874-9236-fdc61b10d692 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:41:52.181392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ad64f9d7-3fe0-4874-9236-fdc61b10d692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:41:52.189739 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 10:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:41:52.225795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54d76e-045c-431b-9703-a7d2afa511f5 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:41:52.237029 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54d76e-045c-431b-9703-a7d2afa511f5 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:41:52.240524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e2400-87b9-4486-acc3-e30506233c41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:41:52.241138 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e2400-87b9-4486-acc3-e30506233c41 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:52.241419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e2400-87b9-4486-acc3-e30506233c41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:52.241729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e2400-87b9-4486-acc3-e30506233c41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:52.250509 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e2400-87b9-4486-acc3-e30506233c41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:41:52.251899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e2400-87b9-4486-acc3-e30506233c41 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:41:52.262238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 759/1533] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:41:55.690520 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85812af-0e49-4f1e-854b-810dc4be3a08 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:55.691458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85812af-0e49-4f1e-854b-810dc4be3a08 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:55.691652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85812af-0e49-4f1e-854b-810dc4be3a08 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:55.693021 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 10:41:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:41:55.698220 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30507e4c-514a-45d8-9d48-b456d7da2fc3 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:55.698684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30507e4c-514a-45d8-9d48-b456d7da2fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:55.698797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30507e4c-514a-45d8-9d48-b456d7da2fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:55.706333 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=81 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:41:55.720625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-30507e4c-514a-45d8-9d48-b456d7da2fc3 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:41:55.721949 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-30507e4c-514a-45d8-9d48-b456d7da2fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:41:55.729921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 10:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:41:56.032794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85812af-0e49-4f1e-854b-810dc4be3a08 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:41:56.034663 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c85812af-0e49-4f1e-854b-810dc4be3a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:41:56.098417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:41:56.111978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85812af-0e49-4f1e-854b-810dc4be3a08 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:41:56.116021 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68a862-020c-49af-b53c-66af591788b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:41:56.116630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68a862-020c-49af-b53c-66af591788b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:56.116909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68a862-020c-49af-b53c-66af591788b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:56.117178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68a862-020c-49af-b53c-66af591788b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:56.125946 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68a862-020c-49af-b53c-66af591788b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:41:56.127332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68a862-020c-49af-b53c-66af591788b5 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:41:56.137598 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 776/1536] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:41:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:41:58.159818 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119550e1-ad2a-4331-8477-f150c42114f0 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:58.160454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119550e1-ad2a-4331-8477-f150c42114f0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:58.160454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119550e1-ad2a-4331-8477-f150c42114f0 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:58.161823 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 10:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:41:58.167454 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85928c9-b234-4f2b-ae75-fe3aeade78e3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:58.167914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85928c9-b234-4f2b-ae75-fe3aeade78e3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:58.168035 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85928c9-b234-4f2b-ae75-fe3aeade78e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:58.188213 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b85928c9-b234-4f2b-ae75-fe3aeade78e3 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:41:58.189429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b85928c9-b234-4f2b-ae75-fe3aeade78e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:41:58.198579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 10:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:41:58.500805 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119550e1-ad2a-4331-8477-f150c42114f0 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:41:58.502688 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-119550e1-ad2a-4331-8477-f150c42114f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:41:58.515601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119550e1-ad2a-4331-8477-f150c42114f0 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:41:58.519111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071d4b5-a579-45ed-8b14-5fd5a980b80c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:41:58.519718 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071d4b5-a579-45ed-8b14-5fd5a980b80c service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:41:58.520002 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071d4b5-a579-45ed-8b14-5fd5a980b80c service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:41:58.520280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071d4b5-a579-45ed-8b14-5fd5a980b80c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:41:58.529141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071d4b5-a579-45ed-8b14-5fd5a980b80c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:41:58.530592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071d4b5-a579-45ed-8b14-5fd5a980b80c service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:41:58.540378 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 762/1539] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:41:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:42:08.183007 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=11 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:42:08.536259 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=8 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:29.662239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b225dbb-e7dc-462c-96d0-75a7d09f2c4d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:29.663032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b225dbb-e7dc-462c-96d0-75a7d09f2c4d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:29.663326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b225dbb-e7dc-462c-96d0-75a7d09f2c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:29.694792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5b225dbb-e7dc-462c-96d0-75a7d09f2c4d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:29.696097 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5b225dbb-e7dc-462c-96d0-75a7d09f2c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:29.712816 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 778/1540] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:29.723138 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32fb03-199e-431f-a6e8-a238b4c7cf33 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:29.724029 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32fb03-199e-431f-a6e8-a238b4c7cf33 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:29.724029 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32fb03-199e-431f-a6e8-a238b4c7cf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:29.759544 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0a32fb03-199e-431f-a6e8-a238b4c7cf33 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:29.760829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0a32fb03-199e-431f-a6e8-a238b4c7cf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:29.769751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 763/1541] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:29.775926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b225dbb-e7dc-462c-96d0-75a7d09f2c4d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:29.804372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b225dbb-e7dc-462c-96d0-75a7d09f2c4d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:29.819310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f33f9-6670-4d60-83f9-66df8e53e9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:29.820404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f33f9-6670-4d60-83f9-66df8e53e9e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:29.820909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f33f9-6670-4d60-83f9-66df8e53e9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:29.821410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f33f9-6670-4d60-83f9-66df8e53e9e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:29.822916 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f33f9-6670-4d60-83f9-66df8e53e9e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:43:29.823692 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f33f9-6670-4d60-83f9-66df8e53e9e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:29.827009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f33f9-6670-4d60-83f9-66df8e53e9e1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:29.855496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 779/1542] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:43:29.864731 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32fb03-199e-431f-a6e8-a238b4c7cf33 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:29.875361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32fb03-199e-431f-a6e8-a238b4c7cf33 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:29.889458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d4556-b4e8-4501-952e-0089755d8e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:29.890163 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d4556-b4e8-4501-952e-0089755d8e68 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:29.890528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d4556-b4e8-4501-952e-0089755d8e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:29.890738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d4556-b4e8-4501-952e-0089755d8e68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:29.891745 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d4556-b4e8-4501-952e-0089755d8e68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:29.893909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d4556-b4e8-4501-952e-0089755d8e68 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:29.967610 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 764/1543] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:43:29 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:43:29.972808 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f33f9-6670-4d60-83f9-66df8e53e9e1 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:30.017657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f33f9-6670-4d60-83f9-66df8e53e9e1 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:30.134247 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8502c-94ba-4f03-8ffc-98dd399066a3 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:30.134619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8502c-94ba-4f03-8ffc-98dd399066a3 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:30.134815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8502c-94ba-4f03-8ffc-98dd399066a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:30.174385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-16e8502c-94ba-4f03-8ffc-98dd399066a3 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:30.177335 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-16e8502c-94ba-4f03-8ffc-98dd399066a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:30.200585 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c997b28-9e22-4ccc-8449-1676d4b2e2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:30.201919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c997b28-9e22-4ccc-8449-1676d4b2e2f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:30.202423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c997b28-9e22-4ccc-8449-1676d4b2e2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:30.203762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c997b28-9e22-4ccc-8449-1676d4b2e2f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:30.205266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c997b28-9e22-4ccc-8449-1676d4b2e2f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:30.209950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c997b28-9e22-4ccc-8449-1676d4b2e2f4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:30.287445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 780/1544] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:43:29 2022] POST /identity/v3/users => generated 293 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:30.289898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14c6d-2219-47db-b7d3-621c335ea03e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:30.290364 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14c6d-2219-47db-b7d3-621c335ea03e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:30.290852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14c6d-2219-47db-b7d3-621c335ea03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:30.330736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 765/1545] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:43:30.331149 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d7d14c6d-2219-47db-b7d3-621c335ea03e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:30.332157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d7d14c6d-2219-47db-b7d3-621c335ea03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:30.333369 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8502c-94ba-4f03-8ffc-98dd399066a3 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:30.357688 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 781/1546] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:30.359195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8637b-b5f7-4a57-9b39-14d94f20be7f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:30.359489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8637b-b5f7-4a57-9b39-14d94f20be7f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:30.359766 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8637b-b5f7-4a57-9b39-14d94f20be7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:30.361560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8502c-94ba-4f03-8ffc-98dd399066a3 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:30.380429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a4aec-5b43-401e-9797-5d62e90e976b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:30.382359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a4aec-5b43-401e-9797-5d62e90e976b admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:30.382914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a4aec-5b43-401e-9797-5d62e90e976b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:30.383756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a4aec-5b43-401e-9797-5d62e90e976b admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:30.392293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a4aec-5b43-401e-9797-5d62e90e976b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:30.394664 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a4aec-5b43-401e-9797-5d62e90e976b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:30.401856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d8e8637b-b5f7-4a57-9b39-14d94f20be7f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:30.406311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d8e8637b-b5f7-4a57-9b39-14d94f20be7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:30.420544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 782/1547] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:30.422127 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c95d8-b0c5-4e29-9794-e569d9aa28ed None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:30.422358 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c95d8-b0c5-4e29-9794-e569d9aa28ed None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:30.422958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c95d8-b0c5-4e29-9794-e569d9aa28ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:30.426677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 766/1548] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:30.428287 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a4aec-5b43-401e-9797-5d62e90e976b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:30.448482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a4aec-5b43-401e-9797-5d62e90e976b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:30.460580 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7291f1-d1f0-4ef2-80f6-6cf5d42efd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:30.462025 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7291f1-d1f0-4ef2-80f6-6cf5d42efd51 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:30.462659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7291f1-d1f0-4ef2-80f6-6cf5d42efd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:30.462904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f38c95d8-b0c5-4e29-9794-e569d9aa28ed None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:30.463402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7291f1-d1f0-4ef2-80f6-6cf5d42efd51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:30.464511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7291f1-d1f0-4ef2-80f6-6cf5d42efd51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:43:30.464636 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f38c95d8-b0c5-4e29-9794-e569d9aa28ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:30.465249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7291f1-d1f0-4ef2-80f6-6cf5d42efd51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:30.466697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: WARNING py.warnings [None req-7f7291f1-d1f0-4ef2-80f6-6cf5d42efd51 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 10:43:30.466697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: warnings.warn(msg) Aug 03 10:43:30.466697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: Aug 03 10:43:30.468143 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7291f1-d1f0-4ef2-80f6-6cf5d42efd51 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:30.485605 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 783/1549] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:30.487028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c95d8-b0c5-4e29-9794-e569d9aa28ed None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:30.506675 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 767/1550] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:43:30.509504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c95d8-b0c5-4e29-9794-e569d9aa28ed None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:30.510686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7291f1-d1f0-4ef2-80f6-6cf5d42efd51 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:30.518901 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68aa0e7-770f-4d1b-ad4e-5e16d0a86836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:30.522675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68aa0e7-770f-4d1b-ad4e-5e16d0a86836 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:30.522675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68aa0e7-770f-4d1b-ad4e-5e16d0a86836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:30.522675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68aa0e7-770f-4d1b-ad4e-5e16d0a86836 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:30.522675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68aa0e7-770f-4d1b-ad4e-5e16d0a86836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:30.523318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68aa0e7-770f-4d1b-ad4e-5e16d0a86836 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:30.532981 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7291f1-d1f0-4ef2-80f6-6cf5d42efd51 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:30.541876 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 784/1551] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:30.542763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68aa0e7-770f-4d1b-ad4e-5e16d0a86836 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:30.547758 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386079b-225e-4294-862b-f27c3424ff69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:30.548819 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386079b-225e-4294-862b-f27c3424ff69 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:30.549326 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386079b-225e-4294-862b-f27c3424ff69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:30.549854 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386079b-225e-4294-862b-f27c3424ff69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:30.550806 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386079b-225e-4294-862b-f27c3424ff69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:43:30.552128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386079b-225e-4294-862b-f27c3424ff69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:30.554204 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386079b-225e-4294-862b-f27c3424ff69 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:30.579295 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68aa0e7-770f-4d1b-ad4e-5e16d0a86836 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:30.587989 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 768/1552] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:43:30.592514 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea5af1-da97-4541-9e11-1ec2fb72e888 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:30.593088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea5af1-da97-4541-9e11-1ec2fb72e888 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:30.593623 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea5af1-da97-4541-9e11-1ec2fb72e888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:30.605102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf07586-d796-4caf-8473-a83be2184689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:30.606671 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf07586-d796-4caf-8473-a83be2184689 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:30.607178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf07586-d796-4caf-8473-a83be2184689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:30.607673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf07586-d796-4caf-8473-a83be2184689 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:30.608919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf07586-d796-4caf-8473-a83be2184689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:30.610868 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: WARNING py.warnings [None req-ebf07586-d796-4caf-8473-a83be2184689 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 10:43:30.610868 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: warnings.warn(msg) Aug 03 10:43:30.610868 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: Aug 03 10:43:30.614678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf07586-d796-4caf-8473-a83be2184689 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:30.670483 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4bea5af1-da97-4541-9e11-1ec2fb72e888 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:30.671787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4bea5af1-da97-4541-9e11-1ec2fb72e888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:30.694007 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 769/1553] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:30.694007 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea5af1-da97-4541-9e11-1ec2fb72e888 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:30.710263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 785/1554] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:43:30 2022] POST /identity/v3/projects => generated 377 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:30.712743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6b2f4-a5ed-4841-aa65-5c9e73f9ef85 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:30.713457 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6b2f4-a5ed-4841-aa65-5c9e73f9ef85 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:30.714161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6b2f4-a5ed-4841-aa65-5c9e73f9ef85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:30.716655 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea5af1-da97-4541-9e11-1ec2fb72e888 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:30.752290 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3db6b2f4-a5ed-4841-aa65-5c9e73f9ef85 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:30.755472 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3db6b2f4-a5ed-4841-aa65-5c9e73f9ef85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:30.803930 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16323c51-239b-4725-9de5-af55072ae495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:30.805567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16323c51-239b-4725-9de5-af55072ae495 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:30.806072 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16323c51-239b-4725-9de5-af55072ae495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:30.806372 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16323c51-239b-4725-9de5-af55072ae495 admin admin] PATH_INFO: `/v3/projects/40540f8322a54133bc32e1af000810ad/users/956aff831fa04d29ba5cffb88e73f4d2/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:30.835782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16323c51-239b-4725-9de5-af55072ae495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40540f8322a54133bc32e1af000810ad, user_id=956aff831fa04d29ba5cffb88e73f4d2, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:30.841084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16323c51-239b-4725-9de5-af55072ae495 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:30.860838 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 786/1555] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:30.862538 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6b2f4-a5ed-4841-aa65-5c9e73f9ef85 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:30.878709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 770/1556] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:30 2022] PUT /identity/v3/projects/40540f8322a54133bc32e1af000810ad/users/956aff831fa04d29ba5cffb88e73f4d2/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:30.881204 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6b2f4-a5ed-4841-aa65-5c9e73f9ef85 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:30.883852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379fd07-b489-4665-ba43-559b68861467 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:30.884840 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379fd07-b489-4665-ba43-559b68861467 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:30.885700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379fd07-b489-4665-ba43-559b68861467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:30.900470 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:30.906894 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:30.923577 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a379fd07-b489-4665-ba43-559b68861467 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:30.927846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a379fd07-b489-4665-ba43-559b68861467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:30.992268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53099a9d-4589-44d0-b88a-56334742a3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:30.992967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53099a9d-4589-44d0-b88a-56334742a3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:30.993248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53099a9d-4589-44d0-b88a-56334742a3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:30.993553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53099a9d-4589-44d0-b88a-56334742a3a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:30.994369 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53099a9d-4589-44d0-b88a-56334742a3a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:30.996354 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53099a9d-4589-44d0-b88a-56334742a3a5 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.030444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 787/1557] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:43:30 2022] POST /identity/v3/projects => generated 391 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:31.032009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec659b3-ee29-4ad1-86d1-93a602e83a11 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.032299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec659b3-ee29-4ad1-86d1-93a602e83a11 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.032758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec659b3-ee29-4ad1-86d1-93a602e83a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.040452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 771/1558] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:31.041803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379fd07-b489-4665-ba43-559b68861467 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.059322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5ec659b3-ee29-4ad1-86d1-93a602e83a11 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:31.059589 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379fd07-b489-4665-ba43-559b68861467 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:31.060563 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5ec659b3-ee29-4ad1-86d1-93a602e83a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:31.143557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 788/1559] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:31.148131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7515b62-b3c0-4bac-b06a-366a8706ff91 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.148131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7515b62-b3c0-4bac-b06a-366a8706ff91 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.148131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7515b62-b3c0-4bac-b06a-366a8706ff91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.149494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075d0d6-cd99-4e2c-993c-aca3fe320af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:31.150267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075d0d6-cd99-4e2c-993c-aca3fe320af4 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.150586 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075d0d6-cd99-4e2c-993c-aca3fe320af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.150896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075d0d6-cd99-4e2c-993c-aca3fe320af4 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.151588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075d0d6-cd99-4e2c-993c-aca3fe320af4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:31.152875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: WARNING py.warnings [None req-e075d0d6-cd99-4e2c-993c-aca3fe320af4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 10:43:31.152875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: warnings.warn(msg) Aug 03 10:43:31.152875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: Aug 03 10:43:31.153926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075d0d6-cd99-4e2c-993c-aca3fe320af4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.171032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c7515b62-b3c0-4bac-b06a-366a8706ff91 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:31.172356 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c7515b62-b3c0-4bac-b06a-366a8706ff91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:31.180608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 772/1560] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:43:31 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:31.181144 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 789/1561] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:31.182505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113e16-925d-42d0-9ae8-593f073ef540 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.182788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113e16-925d-42d0-9ae8-593f073ef540 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.183058 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113e16-925d-42d0-9ae8-593f073ef540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.187344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a186db3-e6f6-4459-b8b7-797945d163e4 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.188030 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a186db3-e6f6-4459-b8b7-797945d163e4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.188504 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a186db3-e6f6-4459-b8b7-797945d163e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.204295 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8c113e16-925d-42d0-9ae8-593f073ef540 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:31.205628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8c113e16-925d-42d0-9ae8-593f073ef540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:31.211864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8a186db3-e6f6-4459-b8b7-797945d163e4 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:31.213321 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8a186db3-e6f6-4459-b8b7-797945d163e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:31.214339 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 790/1562] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:31.215647 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793e83f-13be-456d-8b99-afd441798671 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.215929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793e83f-13be-456d-8b99-afd441798671 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.216204 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793e83f-13be-456d-8b99-afd441798671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.231941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 773/1563] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:31.234552 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a186db3-e6f6-4459-b8b7-797945d163e4 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.241060 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a793e83f-13be-456d-8b99-afd441798671 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:31.242457 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a793e83f-13be-456d-8b99-afd441798671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:31.245183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a186db3-e6f6-4459-b8b7-797945d163e4 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:31.252485 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276033d-ef70-47f5-afaf-8fc608cd08db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:31.253543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276033d-ef70-47f5-afaf-8fc608cd08db admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.254016 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276033d-ef70-47f5-afaf-8fc608cd08db admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.254527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276033d-ef70-47f5-afaf-8fc608cd08db admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.254676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 791/1564] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:31.255778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276033d-ef70-47f5-afaf-8fc608cd08db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:31.256673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047de43-b7b0-448d-b5a9-103a293c0069 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.257119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047de43-b7b0-448d-b5a9-103a293c0069 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.257588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047de43-b7b0-448d-b5a9-103a293c0069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.259545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276033d-ef70-47f5-afaf-8fc608cd08db admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.288630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e047de43-b7b0-448d-b5a9-103a293c0069 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:31.290045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e047de43-b7b0-448d-b5a9-103a293c0069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:31.290719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 774/1565] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:43:31 2022] POST /identity/v3/users => generated 296 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:31.291587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276033d-ef70-47f5-afaf-8fc608cd08db admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.301345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 792/1566] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:31.302000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276033d-ef70-47f5-afaf-8fc608cd08db admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:31.302257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047de43-b7b0-448d-b5a9-103a293c0069 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.307077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe769058-c5db-44ed-b5dd-47ea573f2e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:31.307729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe769058-c5db-44ed-b5dd-47ea573f2e4a admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.308008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe769058-c5db-44ed-b5dd-47ea573f2e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.308282 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe769058-c5db-44ed-b5dd-47ea573f2e4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.308994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe769058-c5db-44ed-b5dd-47ea573f2e4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:43:31.309297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe769058-c5db-44ed-b5dd-47ea573f2e4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:31.311043 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe769058-c5db-44ed-b5dd-47ea573f2e4a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.315890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047de43-b7b0-448d-b5a9-103a293c0069 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:31.320979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f862e20-dc30-4ead-b0c1-de05bd7d3b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:31.321855 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f862e20-dc30-4ead-b0c1-de05bd7d3b73 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.322014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f862e20-dc30-4ead-b0c1-de05bd7d3b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.322216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f862e20-dc30-4ead-b0c1-de05bd7d3b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.322946 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f862e20-dc30-4ead-b0c1-de05bd7d3b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:31.324749 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f862e20-dc30-4ead-b0c1-de05bd7d3b73 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.325459 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 775/1567] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:43:31.328910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bf1db-69b6-490c-a1b5-80b740d3051f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.329515 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bf1db-69b6-490c-a1b5-80b740d3051f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.329877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bf1db-69b6-490c-a1b5-80b740d3051f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.338832 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 793/1568] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:31.339483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f862e20-dc30-4ead-b0c1-de05bd7d3b73 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.358573 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-770bf1db-69b6-490c-a1b5-80b740d3051f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:31.359917 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-770bf1db-69b6-490c-a1b5-80b740d3051f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:31.371788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 776/1569] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:31.373412 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bf1db-69b6-490c-a1b5-80b740d3051f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.388770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bf1db-69b6-490c-a1b5-80b740d3051f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:31.394600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67a8bd-6554-41a5-84a9-7f86150353eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:31.395827 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67a8bd-6554-41a5-84a9-7f86150353eb admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.396117 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67a8bd-6554-41a5-84a9-7f86150353eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.396418 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67a8bd-6554-41a5-84a9-7f86150353eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.397287 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67a8bd-6554-41a5-84a9-7f86150353eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:43:31.397654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67a8bd-6554-41a5-84a9-7f86150353eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:31.399323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67a8bd-6554-41a5-84a9-7f86150353eb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.405953 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f862e20-dc30-4ead-b0c1-de05bd7d3b73 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:31.409950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479118b-bf06-48ff-9771-3c8a1df756b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:31.411148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479118b-bf06-48ff-9771-3c8a1df756b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.411438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479118b-bf06-48ff-9771-3c8a1df756b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.411712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479118b-bf06-48ff-9771-3c8a1df756b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.415453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 777/1570] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:43:31.416853 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de6600-1a43-4745-920e-ec7d1b436c31 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.417143 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de6600-1a43-4745-920e-ec7d1b436c31 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.417621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de6600-1a43-4745-920e-ec7d1b436c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.431768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479118b-bf06-48ff-9771-3c8a1df756b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:31.433424 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479118b-bf06-48ff-9771-3c8a1df756b1 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.440657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-98de6600-1a43-4745-920e-ec7d1b436c31 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:31.442203 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-98de6600-1a43-4745-920e-ec7d1b436c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:31.447677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 794/1571] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:43:31.449244 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479118b-bf06-48ff-9771-3c8a1df756b1 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.455694 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 778/1572] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:31.455694 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de6600-1a43-4745-920e-ec7d1b436c31 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.469296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de6600-1a43-4745-920e-ec7d1b436c31 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:31.474127 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479118b-bf06-48ff-9771-3c8a1df756b1 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:31.479215 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea18714-b3a6-498b-8c66-45e49134be8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:31.479980 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea18714-b3a6-498b-8c66-45e49134be8a admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.480161 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea18714-b3a6-498b-8c66-45e49134be8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.480457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea18714-b3a6-498b-8c66-45e49134be8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.481869 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea18714-b3a6-498b-8c66-45e49134be8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:43:31.482059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea18714-b3a6-498b-8c66-45e49134be8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:31.483105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbe248-e3f0-4e3f-80b6-eb620075aa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:31.483823 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea18714-b3a6-498b-8c66-45e49134be8a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.484158 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbe248-e3f0-4e3f-80b6-eb620075aa8b admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.484645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbe248-e3f0-4e3f-80b6-eb620075aa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.485150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbe248-e3f0-4e3f-80b6-eb620075aa8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.486532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbe248-e3f0-4e3f-80b6-eb620075aa8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:31.490705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbe248-e3f0-4e3f-80b6-eb620075aa8b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.494775 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:31.503286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 779/1573] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:43:31.505107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea18714-b3a6-498b-8c66-45e49134be8a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.518401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea18714-b3a6-498b-8c66-45e49134be8a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:31.528785 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 795/1574] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:31.529114 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2df2d4-dce3-482b-8424-0c8871da97ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:31.529114 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2df2d4-dce3-482b-8424-0c8871da97ea admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.529446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbe248-e3f0-4e3f-80b6-eb620075aa8b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.529662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2df2d4-dce3-482b-8424-0c8871da97ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.529662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2df2d4-dce3-482b-8424-0c8871da97ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.529662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2df2d4-dce3-482b-8424-0c8871da97ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:31.529662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2df2d4-dce3-482b-8424-0c8871da97ea admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.539730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbe248-e3f0-4e3f-80b6-eb620075aa8b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:31.546143 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e51a9-a0f6-4db7-9eb2-facc780cc073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:31.547344 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e51a9-a0f6-4db7-9eb2-facc780cc073 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.547631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e51a9-a0f6-4db7-9eb2-facc780cc073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.547914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e51a9-a0f6-4db7-9eb2-facc780cc073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.548973 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e51a9-a0f6-4db7-9eb2-facc780cc073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:31.550677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e51a9-a0f6-4db7-9eb2-facc780cc073 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.552544 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:31.566199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 796/1575] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:31.567026 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e51a9-a0f6-4db7-9eb2-facc780cc073 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.578369 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 780/1576] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:43:31 2022] POST /identity/v3/projects => generated 375 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:31.578878 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e51a9-a0f6-4db7-9eb2-facc780cc073 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:31.580001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2df2d4-dce3-482b-8424-0c8871da97ea admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.678073 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb7b0d-abe6-4ec9-aad6-8f798c6c4251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:31.678759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb7b0d-abe6-4ec9-aad6-8f798c6c4251 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.679030 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb7b0d-abe6-4ec9-aad6-8f798c6c4251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.679307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb7b0d-abe6-4ec9-aad6-8f798c6c4251 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.680056 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decb7b0d-abe6-4ec9-aad6-8f798c6c4251 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:43:31.680415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decb7b0d-abe6-4ec9-aad6-8f798c6c4251 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:31.682588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decb7b0d-abe6-4ec9-aad6-8f798c6c4251 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.709505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 797/1577] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:43:31.709505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb7b0d-abe6-4ec9-aad6-8f798c6c4251 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.711877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2df2d4-dce3-482b-8424-0c8871da97ea admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:31.711877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bcbe4-34b5-4ca7-81d7-d447bb076575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:31.711877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bcbe4-34b5-4ca7-81d7-d447bb076575 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.711877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bcbe4-34b5-4ca7-81d7-d447bb076575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.722775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bcbe4-34b5-4ca7-81d7-d447bb076575 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.722775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bcbe4-34b5-4ca7-81d7-d447bb076575 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:31.722775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bcbe4-34b5-4ca7-81d7-d447bb076575 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.744877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb7b0d-abe6-4ec9-aad6-8f798c6c4251 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:31.752933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1acd4-93b9-4c92-a82c-41051ea93d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:31.753659 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1acd4-93b9-4c92-a82c-41051ea93d1b admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.753972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1acd4-93b9-4c92-a82c-41051ea93d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.755365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1acd4-93b9-4c92-a82c-41051ea93d1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.756165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1acd4-93b9-4c92-a82c-41051ea93d1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:43:31.756732 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1acd4-93b9-4c92-a82c-41051ea93d1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:31.759520 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1acd4-93b9-4c92-a82c-41051ea93d1b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.771583 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 781/1578] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:43:31 2022] POST /identity/v3/projects => generated 371 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:31.771583 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bcbe4-34b5-4ca7-81d7-d447bb076575 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.781750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 798/1579] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:43:31.783066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1acd4-93b9-4c92-a82c-41051ea93d1b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.803506 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1acd4-93b9-4c92-a82c-41051ea93d1b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:31.875838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bcbe4-34b5-4ca7-81d7-d447bb076575 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:31.882990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4b709-a982-4af6-aa72-4eab7700b0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:31.883651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4b709-a982-4af6-aa72-4eab7700b0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.883929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4b709-a982-4af6-aa72-4eab7700b0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.884203 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4b709-a982-4af6-aa72-4eab7700b0d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.885222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4b709-a982-4af6-aa72-4eab7700b0d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:31.887372 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4b709-a982-4af6-aa72-4eab7700b0d9 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.907761 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b25bd-45a4-4b1d-95ff-6eb56e8a9dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:31.908913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b25bd-45a4-4b1d-95ff-6eb56e8a9dfd admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:31.909391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b25bd-45a4-4b1d-95ff-6eb56e8a9dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:31.909948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b25bd-45a4-4b1d-95ff-6eb56e8a9dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:31.911080 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b25bd-45a4-4b1d-95ff-6eb56e8a9dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:31.913527 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b25bd-45a4-4b1d-95ff-6eb56e8a9dfd admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:31.923211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 782/1580] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:43:31 2022] POST /identity/v3/projects => generated 389 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:31.924592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4b709-a982-4af6-aa72-4eab7700b0d9 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.940430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 799/1581] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:31.948681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b25bd-45a4-4b1d-95ff-6eb56e8a9dfd admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:31.970826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b25bd-45a4-4b1d-95ff-6eb56e8a9dfd admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:32.048093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4b709-a982-4af6-aa72-4eab7700b0d9 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:32.064210 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def8492-f3d4-42a2-bd42-d7c765a9440f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:32.065119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def8492-f3d4-42a2-bd42-d7c765a9440f admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:32.065493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def8492-f3d4-42a2-bd42-d7c765a9440f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:32.065865 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def8492-f3d4-42a2-bd42-d7c765a9440f admin admin] PATH_INFO: `/v3/projects/ab735db52f2e420e9aeba9387484dd07/users/d560ec3b091c4baab93c4abb5fabd287/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:32.075527 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962b6bd-e16f-4633-88a4-761b7941acde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:32.076255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962b6bd-e16f-4633-88a4-761b7941acde admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:32.076608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962b6bd-e16f-4633-88a4-761b7941acde admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:32.076955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962b6bd-e16f-4633-88a4-761b7941acde admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:32.077833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962b6bd-e16f-4633-88a4-761b7941acde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:32.080117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962b6bd-e16f-4633-88a4-761b7941acde admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:32.081302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:32.095552 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:32.100330 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def8492-f3d4-42a2-bd42-d7c765a9440f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab735db52f2e420e9aeba9387484dd07, user_id=d560ec3b091c4baab93c4abb5fabd287, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:32.102734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def8492-f3d4-42a2-bd42-d7c765a9440f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:32.133992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 800/1582] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:43:31 2022] POST /identity/v3/users => generated 288 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:32.133992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962b6bd-e16f-4633-88a4-761b7941acde admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:32.147932 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962b6bd-e16f-4633-88a4-761b7941acde admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:32.148253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 783/1583] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:31 2022] PUT /identity/v3/projects/ab735db52f2e420e9aeba9387484dd07/users/d560ec3b091c4baab93c4abb5fabd287/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:32.148925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def8492-f3d4-42a2-bd42-d7c765a9440f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:32.273185 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a054ad-007b-453d-8b2b-e799ce7e5093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:32.273871 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a054ad-007b-453d-8b2b-e799ce7e5093 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:32.274153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a054ad-007b-453d-8b2b-e799ce7e5093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:32.274426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a054ad-007b-453d-8b2b-e799ce7e5093 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:32.278598 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a054ad-007b-453d-8b2b-e799ce7e5093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:32.280675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a054ad-007b-453d-8b2b-e799ce7e5093 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:32.284097 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def8492-f3d4-42a2-bd42-d7c765a9440f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:32.289213 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbde3fa-6320-4608-85e0-8ac8f0dd6216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:32.290073 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbde3fa-6320-4608-85e0-8ac8f0dd6216 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:32.290462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbde3fa-6320-4608-85e0-8ac8f0dd6216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:32.290844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbde3fa-6320-4608-85e0-8ac8f0dd6216 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:32.291683 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbde3fa-6320-4608-85e0-8ac8f0dd6216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:32.294220 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbde3fa-6320-4608-85e0-8ac8f0dd6216 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:32.322373 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 784/1584] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:43:32 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:32.323347 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbde3fa-6320-4608-85e0-8ac8f0dd6216 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:32.330694 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 801/1585] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:43:32 2022] POST /identity/v3/projects => generated 373 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:32.332093 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a054ad-007b-453d-8b2b-e799ce7e5093 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:32.341728 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbde3fa-6320-4608-85e0-8ac8f0dd6216 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:32.447339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e9bb0-da1f-45c5-8708-f8d96ce9709f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:32.448397 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e9bb0-da1f-45c5-8708-f8d96ce9709f admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:32.448786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e9bb0-da1f-45c5-8708-f8d96ce9709f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:32.449489 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e9bb0-da1f-45c5-8708-f8d96ce9709f admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:32.450424 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e9bb0-da1f-45c5-8708-f8d96ce9709f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:32.453770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e9bb0-da1f-45c5-8708-f8d96ce9709f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:32.463204 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a054ad-007b-453d-8b2b-e799ce7e5093 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:32.470840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13d63e-854c-42ce-89c6-fcd0d2765353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:32.472577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13d63e-854c-42ce-89c6-fcd0d2765353 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:32.473162 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13d63e-854c-42ce-89c6-fcd0d2765353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:32.473779 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13d63e-854c-42ce-89c6-fcd0d2765353 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:32.475296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13d63e-854c-42ce-89c6-fcd0d2765353 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:32.478128 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13d63e-854c-42ce-89c6-fcd0d2765353 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:32.497900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 785/1586] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:43:32 2022] POST /identity/v3/projects => generated 369 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:32.498772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e9bb0-da1f-45c5-8708-f8d96ce9709f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:32.522390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 802/1587] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:43:32 2022] POST /identity/v3/users => generated 295 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:32.523430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13d63e-854c-42ce-89c6-fcd0d2765353 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:32.536040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13d63e-854c-42ce-89c6-fcd0d2765353 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:32.617725 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b073386-0c91-43e1-bb26-c5141e37de35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:32.618276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b073386-0c91-43e1-bb26-c5141e37de35 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:32.618555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b073386-0c91-43e1-bb26-c5141e37de35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:32.618829 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b073386-0c91-43e1-bb26-c5141e37de35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:32.619856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b073386-0c91-43e1-bb26-c5141e37de35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:32.621505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b073386-0c91-43e1-bb26-c5141e37de35 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:32.641482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 803/1588] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:32.643456 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bafd34-cbae-4236-b6fb-718f93639a23 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:32.643896 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bafd34-cbae-4236-b6fb-718f93639a23 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:32.647513 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bafd34-cbae-4236-b6fb-718f93639a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:32.654146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:32.673981 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-45bafd34-cbae-4236-b6fb-718f93639a23 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:32.677225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-45bafd34-cbae-4236-b6fb-718f93639a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:32.691335 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e9bb0-da1f-45c5-8708-f8d96ce9709f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:32.698143 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16afe7f-0687-4e81-a45d-6d13f2e588eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:32.698928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16afe7f-0687-4e81-a45d-6d13f2e588eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:32.699222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16afe7f-0687-4e81-a45d-6d13f2e588eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:32.699450 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 804/1589] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:32.699569 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16afe7f-0687-4e81-a45d-6d13f2e588eb admin admin] PATH_INFO: `/v3/projects/ca8789771e8a4505866424b6c7748291/users/a5e714aad25f4fda8acf767ad7e2490b/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:32.700884 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bafd34-cbae-4236-b6fb-718f93639a23 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:32.710924 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:32.732335 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16afe7f-0687-4e81-a45d-6d13f2e588eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca8789771e8a4505866424b6c7748291, user_id=a5e714aad25f4fda8acf767ad7e2490b, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:32.733238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bafd34-cbae-4236-b6fb-718f93639a23 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:32.735227 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16afe7f-0687-4e81-a45d-6d13f2e588eb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:32.740134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693b736-41a2-407a-8052-808892e4fc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:32.741437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693b736-41a2-407a-8052-808892e4fc9f admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:32.742110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693b736-41a2-407a-8052-808892e4fc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:32.742653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693b736-41a2-407a-8052-808892e4fc9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:32.743729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693b736-41a2-407a-8052-808892e4fc9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:32.746825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693b736-41a2-407a-8052-808892e4fc9f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:32.754612 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 786/1590] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:32 2022] PUT /identity/v3/projects/ca8789771e8a4505866424b6c7748291/users/a5e714aad25f4fda8acf767ad7e2490b/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:32.755981 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16afe7f-0687-4e81-a45d-6d13f2e588eb admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:32.766340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 805/1591] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:32.767918 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693b736-41a2-407a-8052-808892e4fc9f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:32.783595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693b736-41a2-407a-8052-808892e4fc9f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:32.851354 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16afe7f-0687-4e81-a45d-6d13f2e588eb admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:32.857944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3082f9c-02f2-499e-a8b1-b586376d1e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:32.858649 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3082f9c-02f2-499e-a8b1-b586376d1e7f admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:32.859178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3082f9c-02f2-499e-a8b1-b586376d1e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:32.859308 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3082f9c-02f2-499e-a8b1-b586376d1e7f admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:32.860644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3082f9c-02f2-499e-a8b1-b586376d1e7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:32.863619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3082f9c-02f2-499e-a8b1-b586376d1e7f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:32.871866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528388b-9f67-4ec3-a340-ed91bd3fa118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:32.872670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528388b-9f67-4ec3-a340-ed91bd3fa118 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:32.873097 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528388b-9f67-4ec3-a340-ed91bd3fa118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:32.873633 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528388b-9f67-4ec3-a340-ed91bd3fa118 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:32.874897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528388b-9f67-4ec3-a340-ed91bd3fa118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:32.878530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528388b-9f67-4ec3-a340-ed91bd3fa118 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:32.896672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 787/1592] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:43:32 2022] POST /identity/v3/users => generated 287 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:32.899474 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3082f9c-02f2-499e-a8b1-b586376d1e7f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:32.916695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 806/1593] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:43:32 2022] POST /identity/v3/users => generated 285 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:32.917618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528388b-9f67-4ec3-a340-ed91bd3fa118 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:32.924811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3082f9c-02f2-499e-a8b1-b586376d1e7f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:32.929451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528388b-9f67-4ec3-a340-ed91bd3fa118 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:32.934299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37d2bb-0862-41b2-acdf-437e23a16305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:32.935667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ac9ff-5c02-43d8-a132-afa3580c5a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:32.936064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37d2bb-0862-41b2-acdf-437e23a16305 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:32.936320 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ac9ff-5c02-43d8-a132-afa3580c5a9a admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:32.936610 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ac9ff-5c02-43d8-a132-afa3580c5a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:32.936891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ac9ff-5c02-43d8-a132-afa3580c5a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:32.937501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37d2bb-0862-41b2-acdf-437e23a16305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:32.937718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ac9ff-5c02-43d8-a132-afa3580c5a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:32.938508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37d2bb-0862-41b2-acdf-437e23a16305 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:32.939430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ac9ff-5c02-43d8-a132-afa3580c5a9a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:32.939991 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37d2bb-0862-41b2-acdf-437e23a16305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:32.943519 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37d2bb-0862-41b2-acdf-437e23a16305 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:32.957117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 807/1594] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:32.957117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ac9ff-5c02-43d8-a132-afa3580c5a9a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:32.966446 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 788/1595] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:32.969078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aef0f6-fc4a-4be6-91a0-370c82e6c0ba None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:32.970008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aef0f6-fc4a-4be6-91a0-370c82e6c0ba None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:32.970609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aef0f6-fc4a-4be6-91a0-370c82e6c0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:33.002334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-90aef0f6-fc4a-4be6-91a0-370c82e6c0ba None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:33.005578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-90aef0f6-fc4a-4be6-91a0-370c82e6c0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:33.018212 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 789/1596] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:33.019483 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aef0f6-fc4a-4be6-91a0-370c82e6c0ba None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:33.041872 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aef0f6-fc4a-4be6-91a0-370c82e6c0ba None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:33.042134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ac9ff-5c02-43d8-a132-afa3580c5a9a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:33.046870 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68e3da-f1af-4df1-a0bd-7eddd97975e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:33.047752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68e3da-f1af-4df1-a0bd-7eddd97975e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:33.048255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68e3da-f1af-4df1-a0bd-7eddd97975e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:33.048642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68e3da-f1af-4df1-a0bd-7eddd97975e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:33.051792 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb0998-18df-4132-873d-b88f47ff7e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:33.053157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb0998-18df-4132-873d-b88f47ff7e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:33.053720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb0998-18df-4132-873d-b88f47ff7e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:33.054136 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb0998-18df-4132-873d-b88f47ff7e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:33.054997 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb0998-18df-4132-873d-b88f47ff7e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:33.056973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb0998-18df-4132-873d-b88f47ff7e9e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:33.065340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68e3da-f1af-4df1-a0bd-7eddd97975e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:33.067429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68e3da-f1af-4df1-a0bd-7eddd97975e7 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:33.076132 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 790/1597] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:33.078409 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb0998-18df-4132-873d-b88f47ff7e9e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:33.084921 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 808/1598] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:43:33.086654 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68e3da-f1af-4df1-a0bd-7eddd97975e7 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:33.108463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68e3da-f1af-4df1-a0bd-7eddd97975e7 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:33.108930 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb0998-18df-4132-873d-b88f47ff7e9e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:33.116298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c87747-97b8-4e48-a140-6568cec4922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:33.117844 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c87747-97b8-4e48-a140-6568cec4922f admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:33.118134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d1e11-ee3a-4f90-8a76-d58ed023df38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:33.118324 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c87747-97b8-4e48-a140-6568cec4922f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:33.118713 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c87747-97b8-4e48-a140-6568cec4922f admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:33.118910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d1e11-ee3a-4f90-8a76-d58ed023df38 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:33.119176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d1e11-ee3a-4f90-8a76-d58ed023df38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:33.119527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d1e11-ee3a-4f90-8a76-d58ed023df38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:33.119834 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c87747-97b8-4e48-a140-6568cec4922f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:33.120303 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d1e11-ee3a-4f90-8a76-d58ed023df38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:33.122080 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c87747-97b8-4e48-a140-6568cec4922f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:33.122238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d1e11-ee3a-4f90-8a76-d58ed023df38 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:33.138331 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 809/1599] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:33.138331 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c87747-97b8-4e48-a140-6568cec4922f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:33.154239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 791/1600] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:33.154592 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c87747-97b8-4e48-a140-6568cec4922f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:33.155227 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d1e11-ee3a-4f90-8a76-d58ed023df38 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:33.162266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8652e6-bc04-4f16-bf85-8a3790ecbbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:33.163082 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8652e6-bc04-4f16-bf85-8a3790ecbbe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:33.163506 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8652e6-bc04-4f16-bf85-8a3790ecbbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:33.163891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8652e6-bc04-4f16-bf85-8a3790ecbbe6 admin admin] PATH_INFO: `/v3/projects/bcfc3e09588c48299fdf5634cda57416/users/ebdb81686d804eb3864b97481643fc2b/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:33.169188 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d1e11-ee3a-4f90-8a76-d58ed023df38 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:33.174417 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3d58d-dae9-4ee0-baa3-f34019b1cc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:33.175076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3d58d-dae9-4ee0-baa3-f34019b1cc34 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:33.175359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3d58d-dae9-4ee0-baa3-f34019b1cc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:33.175702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3d58d-dae9-4ee0-baa3-f34019b1cc34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:33.176723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc3d58d-dae9-4ee0-baa3-f34019b1cc34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:33.178526 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc3d58d-dae9-4ee0-baa3-f34019b1cc34 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:33.192608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8652e6-bc04-4f16-bf85-8a3790ecbbe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcfc3e09588c48299fdf5634cda57416, user_id=ebdb81686d804eb3864b97481643fc2b, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:33.196791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 792/1601] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:33.196791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3d58d-dae9-4ee0-baa3-f34019b1cc34 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:33.198440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: WARNING py.warnings [None req-0a8652e6-bc04-4f16-bf85-8a3790ecbbe6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 10:43:33.198440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: warnings.warn(msg) Aug 03 10:43:33.198440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: Aug 03 10:43:33.201275 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8652e6-bc04-4f16-bf85-8a3790ecbbe6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:33.206180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3d58d-dae9-4ee0-baa3-f34019b1cc34 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:33.211869 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aedf5f-5338-4f6e-8b7b-e8d39df53e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:33.213259 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aedf5f-5338-4f6e-8b7b-e8d39df53e59 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:33.213259 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aedf5f-5338-4f6e-8b7b-e8d39df53e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:33.213614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aedf5f-5338-4f6e-8b7b-e8d39df53e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:33.226421 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aedf5f-5338-4f6e-8b7b-e8d39df53e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:33.228529 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aedf5f-5338-4f6e-8b7b-e8d39df53e59 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:33.229356 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 810/1602] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:33 2022] PUT /identity/v3/projects/bcfc3e09588c48299fdf5634cda57416/users/ebdb81686d804eb3864b97481643fc2b/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:33.231099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8652e6-bc04-4f16-bf85-8a3790ecbbe6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:33.245017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 793/1603] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:43:33.246145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aedf5f-5338-4f6e-8b7b-e8d39df53e59 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:33.258914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aedf5f-5338-4f6e-8b7b-e8d39df53e59 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:33.333964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:33.389506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50712934-df4f-4c3f-854f-2530309e6411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:33.390442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50712934-df4f-4c3f-854f-2530309e6411 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:33.390738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50712934-df4f-4c3f-854f-2530309e6411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:33.391133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50712934-df4f-4c3f-854f-2530309e6411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:33.391918 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50712934-df4f-4c3f-854f-2530309e6411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:33.393725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50712934-df4f-4c3f-854f-2530309e6411 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:33.422647 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 794/1604] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:33.423673 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50712934-df4f-4c3f-854f-2530309e6411 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:33.435796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8652e6-bc04-4f16-bf85-8a3790ecbbe6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:33.448689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50712934-df4f-4c3f-854f-2530309e6411 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:33.452373 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bdf0d3-cd10-4af9-b146-434530e49158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:33.455159 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdf0d3-cd10-4af9-b146-434530e49158 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:33.455746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdf0d3-cd10-4af9-b146-434530e49158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:33.456939 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdf0d3-cd10-4af9-b146-434530e49158 admin admin] PATH_INFO: `/v3/projects/2f75605309684b169385b6ea4da9f14a/users/22c851fa8799404c9089852e3e6b48e7/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:33.460699 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a478f-f1c2-4e23-89fe-85e13168567f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:33.461664 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a478f-f1c2-4e23-89fe-85e13168567f admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:33.461953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a478f-f1c2-4e23-89fe-85e13168567f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:33.462230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a478f-f1c2-4e23-89fe-85e13168567f admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:33.463893 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a478f-f1c2-4e23-89fe-85e13168567f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:33.464632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a478f-f1c2-4e23-89fe-85e13168567f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:33.466742 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:33.489143 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 795/1605] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:33.496216 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a478f-f1c2-4e23-89fe-85e13168567f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:33.499122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bdf0d3-cd10-4af9-b146-434530e49158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f75605309684b169385b6ea4da9f14a, user_id=22c851fa8799404c9089852e3e6b48e7, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:33.502383 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bdf0d3-cd10-4af9-b146-434530e49158 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:33.524202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a478f-f1c2-4e23-89fe-85e13168567f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:33.538628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 811/1606] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:33 2022] PUT /identity/v3/projects/2f75605309684b169385b6ea4da9f14a/users/22c851fa8799404c9089852e3e6b48e7/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:33.541432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d219e7-bbfa-4dc3-9044-3905f8c7f942 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:33.543147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d219e7-bbfa-4dc3-9044-3905f8c7f942 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:33.543528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d219e7-bbfa-4dc3-9044-3905f8c7f942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:33.592856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c4d219e7-bbfa-4dc3-9044-3905f8c7f942 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:33.596043 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c4d219e7-bbfa-4dc3-9044-3905f8c7f942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:33.687120 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128aa3ee-3605-41f4-a068-a5fa7ad0b447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:33.688479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128aa3ee-3605-41f4-a068-a5fa7ad0b447 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:33.688479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128aa3ee-3605-41f4-a068-a5fa7ad0b447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:33.688479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128aa3ee-3605-41f4-a068-a5fa7ad0b447 admin admin] PATH_INFO: `/v3/projects/157914bad58e4c50b92dfca14ec638aa/users/5744cccc5a224263b71c052dc783e103/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:33.720942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128aa3ee-3605-41f4-a068-a5fa7ad0b447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=157914bad58e4c50b92dfca14ec638aa, user_id=5744cccc5a224263b71c052dc783e103, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:33.724511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128aa3ee-3605-41f4-a068-a5fa7ad0b447 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:33.746512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 796/1607] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:33 2022] PUT /identity/v3/projects/157914bad58e4c50b92dfca14ec638aa/users/5744cccc5a224263b71c052dc783e103/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:33.747026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128aa3ee-3605-41f4-a068-a5fa7ad0b447 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:33.761888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 812/1608] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:33.762944 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d219e7-bbfa-4dc3-9044-3905f8c7f942 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:33.779379 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d219e7-bbfa-4dc3-9044-3905f8c7f942 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:33.911775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128aa3ee-3605-41f4-a068-a5fa7ad0b447 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:33.922040 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e12b3-dc36-40a3-bad8-cc4d3d14b5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:33.923620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e12b3-dc36-40a3-bad8-cc4d3d14b5fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:33.924330 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e12b3-dc36-40a3-bad8-cc4d3d14b5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:33.924692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e12b3-dc36-40a3-bad8-cc4d3d14b5fa admin admin] PATH_INFO: `/v3/projects/c889bda233be4e1ca58f7ee4a8cfc1fb/users/729ce6c25d9b462ea065e5d87acdc39f/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:33.960818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dc1b6-b893-41c1-82d2-c3bc9a899a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:33.961603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dc1b6-b893-41c1-82d2-c3bc9a899a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:33.961891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dc1b6-b893-41c1-82d2-c3bc9a899a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:33.962170 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dc1b6-b893-41c1-82d2-c3bc9a899a7d admin admin] PATH_INFO: `/v3/projects/fd27e5fc185c4d8e8dd91eb172c8257a/users/d743784ca49c4cce8bd9bddba9b19925/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:33.963957 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e12b3-dc36-40a3-bad8-cc4d3d14b5fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c889bda233be4e1ca58f7ee4a8cfc1fb, user_id=729ce6c25d9b462ea065e5d87acdc39f, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:33.969605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e12b3-dc36-40a3-bad8-cc4d3d14b5fa admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:33.997696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dc1b6-b893-41c1-82d2-c3bc9a899a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd27e5fc185c4d8e8dd91eb172c8257a, user_id=d743784ca49c4cce8bd9bddba9b19925, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:34.002474 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dc1b6-b893-41c1-82d2-c3bc9a899a7d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:34.032531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 797/1609] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:33 2022] PUT /identity/v3/projects/c889bda233be4e1ca58f7ee4a8cfc1fb/users/729ce6c25d9b462ea065e5d87acdc39f/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:34.043849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f767f7-4ffc-4f7a-988c-2a89754301e8 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:34.044727 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f767f7-4ffc-4f7a-988c-2a89754301e8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:34.045455 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f767f7-4ffc-4f7a-988c-2a89754301e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:34.067572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 813/1610] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:33 2022] PUT /identity/v3/projects/fd27e5fc185c4d8e8dd91eb172c8257a/users/d743784ca49c4cce8bd9bddba9b19925/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:34.071582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bc6ca-b5f9-409c-b300-57a62d785adc None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:34.072194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bc6ca-b5f9-409c-b300-57a62d785adc None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:34.072626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bc6ca-b5f9-409c-b300-57a62d785adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:34.215792 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c9f767f7-4ffc-4f7a-988c-2a89754301e8 None None] MFA Rules not processed for user `22c851fa8799404c9089852e3e6b48e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:34.215792 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c9f767f7-4ffc-4f7a-988c-2a89754301e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:34.218912 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-176bc6ca-b5f9-409c-b300-57a62d785adc None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:34.218912 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-176bc6ca-b5f9-409c-b300-57a62d785adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:34.277820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:34.320605 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 814/1611] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:34.322300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bc6ca-b5f9-409c-b300-57a62d785adc None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:34.369413 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:34.424796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bc6ca-b5f9-409c-b300-57a62d785adc None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:34.429541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 798/1612] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:34.432269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b1fcd-03ff-4e8b-a86a-147d98e61964 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:34.432663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b1fcd-03ff-4e8b-a86a-147d98e61964 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:34.433264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b1fcd-03ff-4e8b-a86a-147d98e61964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:34.434743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8f1f5-cc53-4276-b158-484a4b25494f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:34.448819 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8f1f5-cc53-4276-b158-484a4b25494f service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:34.449748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8f1f5-cc53-4276-b158-484a4b25494f service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:34.450186 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8f1f5-cc53-4276-b158-484a4b25494f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:34.474146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8f1f5-cc53-4276-b158-484a4b25494f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:34.476639 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8f1f5-cc53-4276-b158-484a4b25494f service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:34.502930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 815/1613] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:43:34.507094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270266b0-1963-4afd-82fd-e46a0f283fa3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:34.507988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270266b0-1963-4afd-82fd-e46a0f283fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:34.507988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270266b0-1963-4afd-82fd-e46a0f283fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:34.547605 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-270266b0-1963-4afd-82fd-e46a0f283fa3 None None] MFA Rules not processed for user `d743784ca49c4cce8bd9bddba9b19925`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:34.549054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-270266b0-1963-4afd-82fd-e46a0f283fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:34.553696 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-369b1fcd-03ff-4e8b-a86a-147d98e61964 None None] MFA Rules not processed for user `729ce6c25d9b462ea065e5d87acdc39f`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:34.560018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-369b1fcd-03ff-4e8b-a86a-147d98e61964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:34.650894 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 816/1614] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:34.652225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270266b0-1963-4afd-82fd-e46a0f283fa3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:34.656188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-270266b0-1963-4afd-82fd-e46a0f283fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:34.684501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270266b0-1963-4afd-82fd-e46a0f283fa3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:34.691618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5396576-427c-4949-a7d3-72fcde45201a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:34.692735 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5396576-427c-4949-a7d3-72fcde45201a service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:34.693094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5396576-427c-4949-a7d3-72fcde45201a service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:34.693533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5396576-427c-4949-a7d3-72fcde45201a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:34.712329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5396576-427c-4949-a7d3-72fcde45201a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:34.715601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5396576-427c-4949-a7d3-72fcde45201a service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:34.735965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 817/1615] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:43:34.737428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5396576-427c-4949-a7d3-72fcde45201a service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:34.750919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5396576-427c-4949-a7d3-72fcde45201a service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:34.763277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a0145-0550-4799-b771-dc549157379d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:34.766613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a0145-0550-4799-b771-dc549157379d admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:34.767807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a0145-0550-4799-b771-dc549157379d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:34.769167 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a0145-0550-4799-b771-dc549157379d admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:34.772122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a0145-0550-4799-b771-dc549157379d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:34.776887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a0145-0550-4799-b771-dc549157379d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:34.796715 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 799/1616] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:34.798923 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7944f0b-12a5-4307-a847-611d1c036e5b None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:34.799648 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7944f0b-12a5-4307-a847-611d1c036e5b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:34.800129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7944f0b-12a5-4307-a847-611d1c036e5b None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:34.802706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 800/1617] 10.222.0.47 () {54 vars in 962 bytes} [Wed Aug 3 10:43:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:43:34.803941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7944f0b-12a5-4307-a847-611d1c036e5b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:34.822303 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7944f0b-12a5-4307-a847-611d1c036e5b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:34.828882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c71a13-1c26-4e86-ab18-429e924314ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:34.830317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c71a13-1c26-4e86-ab18-429e924314ae admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:34.830953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c71a13-1c26-4e86-ab18-429e924314ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:34.831619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c71a13-1c26-4e86-ab18-429e924314ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:34.833632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c71a13-1c26-4e86-ab18-429e924314ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:34.836654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c71a13-1c26-4e86-ab18-429e924314ae admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:34.889696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 818/1618] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:43:34 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:34.895770 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c32f79-91b1-49dc-bc0a-4a8d6e822d3f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:34.896801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c32f79-91b1-49dc-bc0a-4a8d6e822d3f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:34.897457 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c32f79-91b1-49dc-bc0a-4a8d6e822d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:34.924939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 801/1619] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:43:34 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:34.924939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c71a13-1c26-4e86-ab18-429e924314ae admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:35.040617 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-29c32f79-91b1-49dc-bc0a-4a8d6e822d3f None None] MFA Rules not processed for user `e66f25cc329f418eba6bd637bf3e8a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:35.041963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-29c32f79-91b1-49dc-bc0a-4a8d6e822d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:35.047353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c71a13-1c26-4e86-ab18-429e924314ae admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:35.059269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad132541-ee6c-4ee1-bb0d-b615bbf08fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:35.060017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad132541-ee6c-4ee1-bb0d-b615bbf08fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:35.060222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad132541-ee6c-4ee1-bb0d-b615bbf08fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:35.060493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad132541-ee6c-4ee1-bb0d-b615bbf08fc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:35.061201 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad132541-ee6c-4ee1-bb0d-b615bbf08fc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:35.063348 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad132541-ee6c-4ee1-bb0d-b615bbf08fc7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:35.096101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 802/1620] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:43:34 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:35.097086 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad132541-ee6c-4ee1-bb0d-b615bbf08fc7 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:35.114769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad132541-ee6c-4ee1-bb0d-b615bbf08fc7 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:35.122438 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47be61-8966-44bb-998c-9d7640b4d892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:35.123261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47be61-8966-44bb-998c-9d7640b4d892 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:35.123573 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47be61-8966-44bb-998c-9d7640b4d892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:35.123861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47be61-8966-44bb-998c-9d7640b4d892 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:35.124695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47be61-8966-44bb-998c-9d7640b4d892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:35.127563 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47be61-8966-44bb-998c-9d7640b4d892 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:35.148603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:35.160357 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 803/1621] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:43:35 2022] POST /identity/v3/users => generated 291 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:35.161613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47be61-8966-44bb-998c-9d7640b4d892 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:35.183337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47be61-8966-44bb-998c-9d7640b4d892 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:35.190129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e3ce1-175b-4376-a76d-82d61218a670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:35.190980 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e3ce1-175b-4376-a76d-82d61218a670 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:35.191417 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e3ce1-175b-4376-a76d-82d61218a670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:35.192312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e3ce1-175b-4376-a76d-82d61218a670 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:35.193328 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e3ce1-175b-4376-a76d-82d61218a670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:35.195260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e3ce1-175b-4376-a76d-82d61218a670 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:35.211769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 804/1622] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:35.212803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e3ce1-175b-4376-a76d-82d61218a670 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:35.241864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e3ce1-175b-4376-a76d-82d61218a670 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:35.255971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f30e5a-46fc-42b9-88ba-e51844c930e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:35.257445 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f30e5a-46fc-42b9-88ba-e51844c930e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:35.259121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f30e5a-46fc-42b9-88ba-e51844c930e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:35.259121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f30e5a-46fc-42b9-88ba-e51844c930e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:35.260069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f30e5a-46fc-42b9-88ba-e51844c930e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:35.264483 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f30e5a-46fc-42b9-88ba-e51844c930e6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:35.289933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 805/1623] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:35.291674 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f30e5a-46fc-42b9-88ba-e51844c930e6 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:35.313347 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f30e5a-46fc-42b9-88ba-e51844c930e6 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:35.322995 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f2faf-2bbf-4ab5-ae98-493ac396c613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:35.325078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f2faf-2bbf-4ab5-ae98-493ac396c613 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:35.325078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f2faf-2bbf-4ab5-ae98-493ac396c613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:35.325838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f2faf-2bbf-4ab5-ae98-493ac396c613 admin admin] PATH_INFO: `/v3/projects/151af499b5cb4801ace50e3b4c732c10/users/a08a8859e0ee4d1cad4c9ae770b049d0/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:35.335905 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:35.369129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f2faf-2bbf-4ab5-ae98-493ac396c613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=151af499b5cb4801ace50e3b4c732c10, user_id=a08a8859e0ee4d1cad4c9ae770b049d0, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:35.375922 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f2faf-2bbf-4ab5-ae98-493ac396c613 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:35.445408 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 806/1624] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:35 2022] PUT /identity/v3/projects/151af499b5cb4801ace50e3b4c732c10/users/a08a8859e0ee4d1cad4c9ae770b049d0/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:35.448511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f2faf-2bbf-4ab5-ae98-493ac396c613 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:35.476214 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 819/1625] 10.222.0.47 () {58 vars in 1059 bytes} [Wed Aug 3 10:43:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 586 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:35.477595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c32f79-91b1-49dc-bc0a-4a8d6e822d3f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:35.508478 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c32f79-91b1-49dc-bc0a-4a8d6e822d3f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:35.728140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6d833-4045-4d6e-beed-fd657924be53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:35.729013 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6d833-4045-4d6e-beed-fd657924be53 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:35.729408 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6d833-4045-4d6e-beed-fd657924be53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:35.729834 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6d833-4045-4d6e-beed-fd657924be53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:35.731186 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6d833-4045-4d6e-beed-fd657924be53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:35.733014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6d833-4045-4d6e-beed-fd657924be53 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:35.744319 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f2faf-2bbf-4ab5-ae98-493ac396c613 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:35.762494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f37e9-cfc3-4ff2-b13a-b1545aaca98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:35.763280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 820/1626] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:35.764343 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6d833-4045-4d6e-beed-fd657924be53 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:35.767453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f37e9-cfc3-4ff2-b13a-b1545aaca98b admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:35.767802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f37e9-cfc3-4ff2-b13a-b1545aaca98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:35.768081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f37e9-cfc3-4ff2-b13a-b1545aaca98b admin admin] PATH_INFO: `/v3/projects/c9e156da223844aca1059cd0854e046a/users/61186a263a9e43ad92221c8f488d1882/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:35.811026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f37e9-cfc3-4ff2-b13a-b1545aaca98b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e156da223844aca1059cd0854e046a, user_id=61186a263a9e43ad92221c8f488d1882, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:35.814725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f37e9-cfc3-4ff2-b13a-b1545aaca98b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:35.861129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 807/1627] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:35 2022] PUT /identity/v3/projects/c9e156da223844aca1059cd0854e046a/users/61186a263a9e43ad92221c8f488d1882/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:35.863672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f37e9-cfc3-4ff2-b13a-b1545aaca98b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:35.987142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6d833-4045-4d6e-beed-fd657924be53 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:36.048887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f37e9-cfc3-4ff2-b13a-b1545aaca98b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:36.060671 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7d491-05f2-48cf-bd28-b9ec33e8293a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:36.062361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7d491-05f2-48cf-bd28-b9ec33e8293a admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:36.063548 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7d491-05f2-48cf-bd28-b9ec33e8293a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:36.063834 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7d491-05f2-48cf-bd28-b9ec33e8293a admin admin] PATH_INFO: `/v3/projects/151af499b5cb4801ace50e3b4c732c10/users/a08a8859e0ee4d1cad4c9ae770b049d0/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:36.068773 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7d491-05f2-48cf-bd28-b9ec33e8293a admin admin] RBAC: Authorizing `identity:create_grant(project_id=151af499b5cb4801ace50e3b4c732c10, user_id=a08a8859e0ee4d1cad4c9ae770b049d0, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:36.076059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7d491-05f2-48cf-bd28-b9ec33e8293a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:36.116265 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:36.137044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 808/1628] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:35 2022] PUT /identity/v3/projects/151af499b5cb4801ace50e3b4c732c10/users/a08a8859e0ee4d1cad4c9ae770b049d0/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:36.140734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7d491-05f2-48cf-bd28-b9ec33e8293a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:36.170982 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eae922-82ec-45d5-85ab-6dc6ea5c0495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:36.171916 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eae922-82ec-45d5-85ab-6dc6ea5c0495 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:36.172308 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eae922-82ec-45d5-85ab-6dc6ea5c0495 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:36.172696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eae922-82ec-45d5-85ab-6dc6ea5c0495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:36.314765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eae922-82ec-45d5-85ab-6dc6ea5c0495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:36.316575 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eae922-82ec-45d5-85ab-6dc6ea5c0495 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:36.328857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7d491-05f2-48cf-bd28-b9ec33e8293a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:36.340657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c8dd6-7342-488f-957d-da8d183d31db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:36.343195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c8dd6-7342-488f-957d-da8d183d31db admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:36.343439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c8dd6-7342-488f-957d-da8d183d31db admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:36.343858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c8dd6-7342-488f-957d-da8d183d31db admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:36.344053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 821/1629] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:43:36.345372 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c8dd6-7342-488f-957d-da8d183d31db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:36.348795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c8dd6-7342-488f-957d-da8d183d31db admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:36.349501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd76d0a-ca56-439d-bd84-28bf3bf918e3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:36.349704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd76d0a-ca56-439d-bd84-28bf3bf918e3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:36.350024 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd76d0a-ca56-439d-bd84-28bf3bf918e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:36.353258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:36.377713 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 809/1630] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:36.385942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c8dd6-7342-488f-957d-da8d183d31db admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:36.408885 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c8dd6-7342-488f-957d-da8d183d31db admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:36.419181 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec663903-92f3-49c5-8260-ce310fd9aa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:36.422075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec663903-92f3-49c5-8260-ce310fd9aa7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:36.422534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec663903-92f3-49c5-8260-ce310fd9aa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:36.423026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec663903-92f3-49c5-8260-ce310fd9aa7f admin admin] PATH_INFO: `/v3/projects/c9e156da223844aca1059cd0854e046a/users/61186a263a9e43ad92221c8f488d1882/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:36.428872 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec663903-92f3-49c5-8260-ce310fd9aa7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e156da223844aca1059cd0854e046a, user_id=61186a263a9e43ad92221c8f488d1882, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:36.434232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec663903-92f3-49c5-8260-ce310fd9aa7f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:36.454310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-efd76d0a-ca56-439d-bd84-28bf3bf918e3 None None] MFA Rules not processed for user `a08a8859e0ee4d1cad4c9ae770b049d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:36.455693 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-efd76d0a-ca56-439d-bd84-28bf3bf918e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:36.474524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 810/1631] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:36 2022] PUT /identity/v3/projects/c9e156da223844aca1059cd0854e046a/users/61186a263a9e43ad92221c8f488d1882/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:36.484872 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890e631-290d-47be-9bb3-35c34a535aa4 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:36.485371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890e631-290d-47be-9bb3-35c34a535aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:36.486246 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890e631-290d-47be-9bb3-35c34a535aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:36.578488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-6890e631-290d-47be-9bb3-35c34a535aa4 None None] MFA Rules not processed for user `61186a263a9e43ad92221c8f488d1882`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:36.581420 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6890e631-290d-47be-9bb3-35c34a535aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:36.698121 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 822/1632] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:36.732651 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd76d0a-ca56-439d-bd84-28bf3bf918e3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:36.873671 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd76d0a-ca56-439d-bd84-28bf3bf918e3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:36.875905 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 811/1633] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:36.884152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5237cf-44b0-4a92-9e0e-3fe554d13455 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:36.885657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5237cf-44b0-4a92-9e0e-3fe554d13455 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:36.885657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5237cf-44b0-4a92-9e0e-3fe554d13455 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:36.885657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5237cf-44b0-4a92-9e0e-3fe554d13455 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:36.892749 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890e631-290d-47be-9bb3-35c34a535aa4 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:36.912764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5237cf-44b0-4a92-9e0e-3fe554d13455 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:36.914359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5237cf-44b0-4a92-9e0e-3fe554d13455 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:36.924705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890e631-290d-47be-9bb3-35c34a535aa4 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:36.934360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a8dc0-6b2d-4d5c-9dab-92ac34f48945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:36.935254 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a8dc0-6b2d-4d5c-9dab-92ac34f48945 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:36.936036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a8dc0-6b2d-4d5c-9dab-92ac34f48945 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:36.936956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 823/1634] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:43:36.948639 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a8dc0-6b2d-4d5c-9dab-92ac34f48945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:36.995839 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a8dc0-6b2d-4d5c-9dab-92ac34f48945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:36.997699 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a8dc0-6b2d-4d5c-9dab-92ac34f48945 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:37.048452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 812/1635] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:36 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:43:37.134128 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e06ff-ce52-490f-8165-595a129e0dac None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:37.134741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e06ff-ce52-490f-8165-595a129e0dac None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:37.135184 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e06ff-ce52-490f-8165-595a129e0dac None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:37.136851 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 824/1636] 10.222.0.47 () {54 vars in 912 bytes} [Wed Aug 3 10:43:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:43:37.149750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7a313-2249-440f-8b50-cd568cc18040 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:37.151015 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7a313-2249-440f-8b50-cd568cc18040 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:37.151618 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7a313-2249-440f-8b50-cd568cc18040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:37.165677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e06ff-ce52-490f-8165-595a129e0dac None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:37.187844 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e06ff-ce52-490f-8165-595a129e0dac None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:37.192376 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f71a1-6402-48c9-9e6b-3513fc7a0ca0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:37.193022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f71a1-6402-48c9-9e6b-3513fc7a0ca0 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:37.193302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f71a1-6402-48c9-9e6b-3513fc7a0ca0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:37.193603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f71a1-6402-48c9-9e6b-3513fc7a0ca0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:37.203194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d9d7a313-2249-440f-8b50-cd568cc18040 None None] MFA Rules not processed for user `e66f25cc329f418eba6bd637bf3e8a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:37.206454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d9d7a313-2249-440f-8b50-cd568cc18040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:37.229201 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:37.346004 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f71a1-6402-48c9-9e6b-3513fc7a0ca0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:37.349395 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f71a1-6402-48c9-9e6b-3513fc7a0ca0 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:37.350690 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 813/1637] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 3 10:43:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:37.352192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7a313-2249-440f-8b50-cd568cc18040 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:37.370620 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 825/1638] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:43:37.390124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7a313-2249-440f-8b50-cd568cc18040 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:37.398509 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad640c0-2e9c-45c9-b073-42cb58fb4217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:37.399799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad640c0-2e9c-45c9-b073-42cb58fb4217 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:37.400349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad640c0-2e9c-45c9-b073-42cb58fb4217 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:37.400877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad640c0-2e9c-45c9-b073-42cb58fb4217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:37.498266 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:37.526022 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad640c0-2e9c-45c9-b073-42cb58fb4217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:37.528145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad640c0-2e9c-45c9-b073-42cb58fb4217 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:37.543570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 814/1639] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:43:38.111899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f71a1-6402-48c9-9e6b-3513fc7a0ca0 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:38.131726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f71a1-6402-48c9-9e6b-3513fc7a0ca0 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:38.144021 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43fc4b-2c6a-4bab-ac87-e64fce005d92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:38.147282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43fc4b-2c6a-4bab-ac87-e64fce005d92 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:38.147962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43fc4b-2c6a-4bab-ac87-e64fce005d92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:38.150076 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43fc4b-2c6a-4bab-ac87-e64fce005d92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:38.180088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43fc4b-2c6a-4bab-ac87-e64fce005d92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:38.191289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43fc4b-2c6a-4bab-ac87-e64fce005d92 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:38.230553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 826/1640] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:43:39.926751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc67d6-29a1-4a5d-bb4e-b4b21bf3429f None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:39.927441 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc67d6-29a1-4a5d-bb4e-b4b21bf3429f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:39.928064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc67d6-29a1-4a5d-bb4e-b4b21bf3429f None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:39.932131 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 815/1641] 10.222.0.47 () {54 vars in 962 bytes} [Wed Aug 3 10:43:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:43:39.943082 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c2a19-dbfc-46ad-a7c9-792e5d882978 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:39.943683 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c2a19-dbfc-46ad-a7c9-792e5d882978 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:39.944270 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c2a19-dbfc-46ad-a7c9-792e5d882978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:40.017479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-291c2a19-dbfc-46ad-a7c9-792e5d882978 None None] MFA Rules not processed for user `e66f25cc329f418eba6bd637bf3e8a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:40.019726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-291c2a19-dbfc-46ad-a7c9-792e5d882978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:40.080882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 827/1642] 10.222.0.47 () {58 vars in 1059 bytes} [Wed Aug 3 10:43:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:40.088361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b6b53-dba7-45f0-82d9-99eb8b6cade1 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:40.088361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b6b53-dba7-45f0-82d9-99eb8b6cade1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:40.088361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b6b53-dba7-45f0-82d9-99eb8b6cade1 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:40.088361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 816/1643] 10.222.0.47 () {54 vars in 912 bytes} [Wed Aug 3 10:43:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:43:40.089096 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbb399-080a-4c57-90ff-847626bb1ac5 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:40.089096 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbb399-080a-4c57-90ff-847626bb1ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:40.089096 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbb399-080a-4c57-90ff-847626bb1ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:40.163091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f9fbb399-080a-4c57-90ff-847626bb1ac5 None None] MFA Rules not processed for user `e66f25cc329f418eba6bd637bf3e8a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:40.164477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f9fbb399-080a-4c57-90ff-847626bb1ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:40.189884 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 828/1644] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 3 10:43:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 03 10:43:40.205902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b6b53-dba7-45f0-82d9-99eb8b6cade1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:40.226330 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b6b53-dba7-45f0-82d9-99eb8b6cade1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:40.240200 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5bfcc-fed9-4ac5-846e-3b0d3886733b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:40.241875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5bfcc-fed9-4ac5-846e-3b0d3886733b service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:40.242409 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5bfcc-fed9-4ac5-846e-3b0d3886733b service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:40.242887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5bfcc-fed9-4ac5-846e-3b0d3886733b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:40.259220 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5bfcc-fed9-4ac5-846e-3b0d3886733b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:40.261957 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5bfcc-fed9-4ac5-846e-3b0d3886733b service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:40.297043 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 817/1645] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:43:42.612733 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbb399-080a-4c57-90ff-847626bb1ac5 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:42.853000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbb399-080a-4c57-90ff-847626bb1ac5 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:42.898103 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98824ec1-4637-4e4b-9466-cec03155f04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:42.909791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98824ec1-4637-4e4b-9466-cec03155f04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:42.911040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98824ec1-4637-4e4b-9466-cec03155f04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:42.911787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98824ec1-4637-4e4b-9466-cec03155f04b admin admin] PATH_INFO: `/v3/users/d743784ca49c4cce8bd9bddba9b19925` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:42.914848 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98824ec1-4637-4e4b-9466-cec03155f04b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d743784ca49c4cce8bd9bddba9b19925)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:42.917312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: WARNING py.warnings [None req-98824ec1-4637-4e4b-9466-cec03155f04b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 10:43:42.917312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: warnings.warn(msg) Aug 03 10:43:42.917312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: Aug 03 10:43:42.922262 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98824ec1-4637-4e4b-9466-cec03155f04b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:43.206735 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-98824ec1-4637-4e4b-9466-cec03155f04b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd743784ca49c4cce8bd9bddba9b19925'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:43.266427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-98824ec1-4637-4e4b-9466-cec03155f04b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd743784ca49c4cce8bd9bddba9b19925'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:43.267961 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-98824ec1-4637-4e4b-9466-cec03155f04b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd743784ca49c4cce8bd9bddba9b19925'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:43.295505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-98824ec1-4637-4e4b-9466-cec03155f04b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd743784ca49c4cce8bd9bddba9b19925'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:43.314674 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-98824ec1-4637-4e4b-9466-cec03155f04b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd743784ca49c4cce8bd9bddba9b19925'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:43.334071 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 829/1646] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:43:42 2022] DELETE /identity/v3/users/d743784ca49c4cce8bd9bddba9b19925 => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:43.359838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58e17c-c17f-4918-a70d-9c52026b61ca None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:43.361037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58e17c-c17f-4918-a70d-9c52026b61ca None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:43.361763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58e17c-c17f-4918-a70d-9c52026b61ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:43.410220 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3a58e17c-c17f-4918-a70d-9c52026b61ca None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:43.414978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3a58e17c-c17f-4918-a70d-9c52026b61ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:43.714375 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 818/1647] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:43.748748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98824ec1-4637-4e4b-9466-cec03155f04b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:43.750546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-98824ec1-4637-4e4b-9466-cec03155f04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:43.772758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:43.948765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98824ec1-4637-4e4b-9466-cec03155f04b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:43.965778 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a28f0-f501-41ae-abbd-0b02f2e0a900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:43.966846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a28f0-f501-41ae-abbd-0b02f2e0a900 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:43.966999 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a28f0-f501-41ae-abbd-0b02f2e0a900 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:43.967247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a28f0-f501-41ae-abbd-0b02f2e0a900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:43.992510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a28f0-f501-41ae-abbd-0b02f2e0a900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:43.994862 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a28f0-f501-41ae-abbd-0b02f2e0a900 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:44.014609 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 830/1648] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:43:46.030963 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58e17c-c17f-4918-a70d-9c52026b61ca None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:46.033302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3a58e17c-c17f-4918-a70d-9c52026b61ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:46.059288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58e17c-c17f-4918-a70d-9c52026b61ca None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:46.082083 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25295a27-b26d-47ac-b5e5-3309353453bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:46.083497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25295a27-b26d-47ac-b5e5-3309353453bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:46.083759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25295a27-b26d-47ac-b5e5-3309353453bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:46.084200 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25295a27-b26d-47ac-b5e5-3309353453bb admin admin] PATH_INFO: `/v3/users/729ce6c25d9b462ea065e5d87acdc39f` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:46.086329 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25295a27-b26d-47ac-b5e5-3309353453bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=729ce6c25d9b462ea065e5d87acdc39f)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:46.088943 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25295a27-b26d-47ac-b5e5-3309353453bb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:46.221680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-25295a27-b26d-47ac-b5e5-3309353453bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '729ce6c25d9b462ea065e5d87acdc39f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:46.246629 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-25295a27-b26d-47ac-b5e5-3309353453bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '729ce6c25d9b462ea065e5d87acdc39f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:46.275058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-25295a27-b26d-47ac-b5e5-3309353453bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '729ce6c25d9b462ea065e5d87acdc39f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:46.275874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-25295a27-b26d-47ac-b5e5-3309353453bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '729ce6c25d9b462ea065e5d87acdc39f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:46.276575 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-25295a27-b26d-47ac-b5e5-3309353453bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '729ce6c25d9b462ea065e5d87acdc39f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:46.324923 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 819/1649] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:43:46 2022] DELETE /identity/v3/users/729ce6c25d9b462ea065e5d87acdc39f => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:46.331219 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ac61a-6d7e-44f6-8479-f5ed8b4ebb8a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:46.333560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ac61a-6d7e-44f6-8479-f5ed8b4ebb8a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:46.334010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ac61a-6d7e-44f6-8479-f5ed8b4ebb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:46.400724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1a6ac61a-6d7e-44f6-8479-f5ed8b4ebb8a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:46.403990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1a6ac61a-6d7e-44f6-8479-f5ed8b4ebb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:46.608600 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 831/1650] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:43:46.692191 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25295a27-b26d-47ac-b5e5-3309353453bb admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:46.695474 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-25295a27-b26d-47ac-b5e5-3309353453bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:46.906940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:46.941448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25295a27-b26d-47ac-b5e5-3309353453bb admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:46.961510 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd8055-8160-4996-94b0-a29c8d6c2134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:46.963028 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd8055-8160-4996-94b0-a29c8d6c2134 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:46.963752 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd8055-8160-4996-94b0-a29c8d6c2134 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:46.964479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd8055-8160-4996-94b0-a29c8d6c2134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:47.026402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd8055-8160-4996-94b0-a29c8d6c2134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:47.033703 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd8055-8160-4996-94b0-a29c8d6c2134 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:47.092570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 820/1651] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:43:49.178346 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ac61a-6d7e-44f6-8479-f5ed8b4ebb8a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:49.181352 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1a6ac61a-6d7e-44f6-8479-f5ed8b4ebb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:49.241006 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ac61a-6d7e-44f6-8479-f5ed8b4ebb8a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:49.252065 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8286c-84bb-4dc3-bdbf-ae5db25305d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:49.253155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8286c-84bb-4dc3-bdbf-ae5db25305d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:49.254868 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8286c-84bb-4dc3-bdbf-ae5db25305d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:49.257623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8286c-84bb-4dc3-bdbf-ae5db25305d1 admin admin] PATH_INFO: `/v3/projects/fd27e5fc185c4d8e8dd91eb172c8257a` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:49.261656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8286c-84bb-4dc3-bdbf-ae5db25305d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd27e5fc185c4d8e8dd91eb172c8257a)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:49.274891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8286c-84bb-4dc3-bdbf-ae5db25305d1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:49.416425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-60c8286c-84bb-4dc3-bdbf-ae5db25305d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd27e5fc185c4d8e8dd91eb172c8257a', 'request_id': 'req-60c8286c-84bb-4dc3-bdbf-ae5db25305d1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:49.441767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-60c8286c-84bb-4dc3-bdbf-ae5db25305d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd27e5fc185c4d8e8dd91eb172c8257a', 'request_id': 'req-60c8286c-84bb-4dc3-bdbf-ae5db25305d1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:49.452784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-60c8286c-84bb-4dc3-bdbf-ae5db25305d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd27e5fc185c4d8e8dd91eb172c8257a', 'request_id': 'req-60c8286c-84bb-4dc3-bdbf-ae5db25305d1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:49.484442 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-60c8286c-84bb-4dc3-bdbf-ae5db25305d1 admin admin] The token cache is being invalidate because project fd27e5fc185c4d8e8dd91eb172c8257a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:43:49.484761 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-60c8286c-84bb-4dc3-bdbf-ae5db25305d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:49.486858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-60c8286c-84bb-4dc3-bdbf-ae5db25305d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:49.497640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 832/1652] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:43:49 2022] DELETE /identity/v3/projects/fd27e5fc185c4d8e8dd91eb172c8257a => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:49.501310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c90991-fec5-4d6f-af66-4d5de8b4d517 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:49.502096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c90991-fec5-4d6f-af66-4d5de8b4d517 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:49.502734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c90991-fec5-4d6f-af66-4d5de8b4d517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:49.606413 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c0c90991-fec5-4d6f-af66-4d5de8b4d517 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:49.612542 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c0c90991-fec5-4d6f-af66-4d5de8b4d517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:49.704682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6165a2c-2f64-46a6-9eeb-89af5158e133 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:49.704956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6165a2c-2f64-46a6-9eeb-89af5158e133 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:49.705240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6165a2c-2f64-46a6-9eeb-89af5158e133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:49.793589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f6165a2c-2f64-46a6-9eeb-89af5158e133 None None] MFA Rules not processed for user `ebdb81686d804eb3864b97481643fc2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:49.795141 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f6165a2c-2f64-46a6-9eeb-89af5158e133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:49.851736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:49.893422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 821/1653] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:49.902657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9c10e-9b00-49d7-8718-a0547bba5af8 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:49.903398 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9c10e-9b00-49d7-8718-a0547bba5af8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:49.904053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9c10e-9b00-49d7-8718-a0547bba5af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:50.009231 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 833/1654] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 03 10:43:50.016774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-eed9c10e-9b00-49d7-8718-a0547bba5af8 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:50.024959 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-eed9c10e-9b00-49d7-8718-a0547bba5af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:50.025883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6165a2c-2f64-46a6-9eeb-89af5158e133 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:50.027673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f6165a2c-2f64-46a6-9eeb-89af5158e133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:50.058848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 822/1655] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:43:50.063419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9c10e-9b00-49d7-8718-a0547bba5af8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:50.098614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9c10e-9b00-49d7-8718-a0547bba5af8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:50.136156 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643ac1d-6ccb-4b95-8053-e9bfa085904b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:50.137610 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643ac1d-6ccb-4b95-8053-e9bfa085904b admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:50.138226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643ac1d-6ccb-4b95-8053-e9bfa085904b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:50.138761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643ac1d-6ccb-4b95-8053-e9bfa085904b admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:50.140089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643ac1d-6ccb-4b95-8053-e9bfa085904b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:43:50.140686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643ac1d-6ccb-4b95-8053-e9bfa085904b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:50.143902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643ac1d-6ccb-4b95-8053-e9bfa085904b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:50.191711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 823/1656] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:43:50.197344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643ac1d-6ccb-4b95-8053-e9bfa085904b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:50.222705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6165a2c-2f64-46a6-9eeb-89af5158e133 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:50.237702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571a254-4f28-49f1-82cc-6744e50c2ae7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:50.238520 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571a254-4f28-49f1-82cc-6744e50c2ae7 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:50.239309 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571a254-4f28-49f1-82cc-6744e50c2ae7 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:50.239667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571a254-4f28-49f1-82cc-6744e50c2ae7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:50.241684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643ac1d-6ccb-4b95-8053-e9bfa085904b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:50.262870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d0c53-02ac-478e-b8ea-d08401ae556f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:50.264110 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d0c53-02ac-478e-b8ea-d08401ae556f admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:50.264681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d0c53-02ac-478e-b8ea-d08401ae556f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:50.264795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571a254-4f28-49f1-82cc-6744e50c2ae7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:50.265287 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d0c53-02ac-478e-b8ea-d08401ae556f admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:50.266655 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d0c53-02ac-478e-b8ea-d08401ae556f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:50.273926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d0c53-02ac-478e-b8ea-d08401ae556f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:50.280180 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571a254-4f28-49f1-82cc-6744e50c2ae7 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:50.294541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 834/1657] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:43:50.389632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571a254-4f28-49f1-82cc-6744e50c2ae7 service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:50.395975 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 824/1658] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:43:50 2022] POST /identity/v3/projects => generated 381 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:50.425146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571a254-4f28-49f1-82cc-6744e50c2ae7 service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:50.638400 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27a8dd-a608-4e3e-a4c5-a054b39b33b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:50.642393 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27a8dd-a608-4e3e-a4c5-a054b39b33b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:50.656933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27a8dd-a608-4e3e-a4c5-a054b39b33b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:50.657622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27a8dd-a608-4e3e-a4c5-a054b39b33b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:50.662560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e27a8dd-a608-4e3e-a4c5-a054b39b33b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:50.670507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e27a8dd-a608-4e3e-a4c5-a054b39b33b6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:50.769197 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d0c53-02ac-478e-b8ea-d08401ae556f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:50.784780 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 835/1659] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:43:50 2022] POST /identity/v3/users => generated 291 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:50.788262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d0c53-02ac-478e-b8ea-d08401ae556f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:50.794549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c07555-90e4-4eb8-b898-41ba93077895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:50.795619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c07555-90e4-4eb8-b898-41ba93077895 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:50.796136 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c07555-90e4-4eb8-b898-41ba93077895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:50.796645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c07555-90e4-4eb8-b898-41ba93077895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:50.797680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c07555-90e4-4eb8-b898-41ba93077895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:50.801877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c07555-90e4-4eb8-b898-41ba93077895 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:50.826112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27a8dd-a608-4e3e-a4c5-a054b39b33b6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:50.838249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 825/1660] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:43:50.859240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27a8dd-a608-4e3e-a4c5-a054b39b33b6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:50.876952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79e96d-3660-4a2e-b021-b8b70e62ea6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:50.879753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79e96d-3660-4a2e-b021-b8b70e62ea6e admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:50.883724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79e96d-3660-4a2e-b021-b8b70e62ea6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:50.886848 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79e96d-3660-4a2e-b021-b8b70e62ea6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:50.888748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79e96d-3660-4a2e-b021-b8b70e62ea6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:50.894034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79e96d-3660-4a2e-b021-b8b70e62ea6e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:50.932030 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 836/1661] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:50.932337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c07555-90e4-4eb8-b898-41ba93077895 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:50.938965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79e96d-3660-4a2e-b021-b8b70e62ea6e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:50.955036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c07555-90e4-4eb8-b898-41ba93077895 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:50.955143 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-cf79e96d-3660-4a2e-b021-b8b70e62ea6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:50.971157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b1444-48de-47e9-b630-6fa5e1cae3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:50.972327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b1444-48de-47e9-b630-6fa5e1cae3c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:50.973923 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b1444-48de-47e9-b630-6fa5e1cae3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:50.974672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b1444-48de-47e9-b630-6fa5e1cae3c6 admin admin] PATH_INFO: `/v3/projects/064ea3050ab546439218d9a1e8742e19/users/82c9f163a3764a30b7f938b362900b69/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:50.995831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79e96d-3660-4a2e-b021-b8b70e62ea6e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:51.015702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78f306-f460-49ac-9244-ed46517aae53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:51.016412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78f306-f460-49ac-9244-ed46517aae53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:51.016716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78f306-f460-49ac-9244-ed46517aae53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:51.017003 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78f306-f460-49ac-9244-ed46517aae53 admin admin] PATH_INFO: `/v3/projects/c889bda233be4e1ca58f7ee4a8cfc1fb` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:51.021268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78f306-f460-49ac-9244-ed46517aae53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c889bda233be4e1ca58f7ee4a8cfc1fb)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:51.035366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78f306-f460-49ac-9244-ed46517aae53 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:51.044368 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b1444-48de-47e9-b630-6fa5e1cae3c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=064ea3050ab546439218d9a1e8742e19, user_id=82c9f163a3764a30b7f938b362900b69, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:51.074857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b1444-48de-47e9-b630-6fa5e1cae3c6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:51.110878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 826/1662] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:50 2022] PUT /identity/v3/projects/064ea3050ab546439218d9a1e8742e19/users/82c9f163a3764a30b7f938b362900b69/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:51.129784 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dde9b-0466-43d6-8b46-3e8ceb85f895 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:51.131047 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dde9b-0466-43d6-8b46-3e8ceb85f895 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:51.131893 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dde9b-0466-43d6-8b46-3e8ceb85f895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:51.213400 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:51.229506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-188dde9b-0466-43d6-8b46-3e8ceb85f895 None None] MFA Rules not processed for user `82c9f163a3764a30b7f938b362900b69`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:51.230592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-188dde9b-0466-43d6-8b46-3e8ceb85f895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:51.315726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-aa78f306-f460-49ac-9244-ed46517aae53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c889bda233be4e1ca58f7ee4a8cfc1fb', 'request_id': 'req-aa78f306-f460-49ac-9244-ed46517aae53'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:51.341516 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-aa78f306-f460-49ac-9244-ed46517aae53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c889bda233be4e1ca58f7ee4a8cfc1fb', 'request_id': 'req-aa78f306-f460-49ac-9244-ed46517aae53'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:51.355229 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-aa78f306-f460-49ac-9244-ed46517aae53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c889bda233be4e1ca58f7ee4a8cfc1fb', 'request_id': 'req-aa78f306-f460-49ac-9244-ed46517aae53'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:51.384219 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-aa78f306-f460-49ac-9244-ed46517aae53 admin admin] The token cache is being invalidate because project c889bda233be4e1ca58f7ee4a8cfc1fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:43:51.384809 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-aa78f306-f460-49ac-9244-ed46517aae53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:51.386203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-aa78f306-f460-49ac-9244-ed46517aae53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:51.388506 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 837/1663] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:43:50 2022] DELETE /identity/v3/projects/c889bda233be4e1ca58f7ee4a8cfc1fb => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:51.398050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78f306-f460-49ac-9244-ed46517aae53 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:51.401977 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-aa78f306-f460-49ac-9244-ed46517aae53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:51.443518 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 827/1664] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:51.478002 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dde9b-0466-43d6-8b46-3e8ceb85f895 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:51.486570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-188dde9b-0466-43d6-8b46-3e8ceb85f895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:51.527711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:51.626406 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78f306-f460-49ac-9244-ed46517aae53 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:51.650448 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743630a-a9b3-45db-a891-e53152a4eba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:51.651181 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743630a-a9b3-45db-a891-e53152a4eba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:51.651465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743630a-a9b3-45db-a891-e53152a4eba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:51.652437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743630a-a9b3-45db-a891-e53152a4eba2 admin admin] PATH_INFO: `/v3/users/61186a263a9e43ad92221c8f488d1882` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:51.653870 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743630a-a9b3-45db-a891-e53152a4eba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61186a263a9e43ad92221c8f488d1882)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:51.655845 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743630a-a9b3-45db-a891-e53152a4eba2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:51.811837 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e743630a-a9b3-45db-a891-e53152a4eba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61186a263a9e43ad92221c8f488d1882'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:51.870452 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e743630a-a9b3-45db-a891-e53152a4eba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61186a263a9e43ad92221c8f488d1882'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:51.876598 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e743630a-a9b3-45db-a891-e53152a4eba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61186a263a9e43ad92221c8f488d1882'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:51.898472 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e743630a-a9b3-45db-a891-e53152a4eba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61186a263a9e43ad92221c8f488d1882'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:51.923944 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e743630a-a9b3-45db-a891-e53152a4eba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61186a263a9e43ad92221c8f488d1882'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:51.949150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 838/1665] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:43:51 2022] DELETE /identity/v3/users/61186a263a9e43ad92221c8f488d1882 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:51.954636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dde9b-0466-43d6-8b46-3e8ceb85f895 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:52.223735 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903dcb3c-60b9-4b64-9329-eb1bfa52a81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:52.224495 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903dcb3c-60b9-4b64-9329-eb1bfa52a81c admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:52.224808 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903dcb3c-60b9-4b64-9329-eb1bfa52a81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:52.225217 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903dcb3c-60b9-4b64-9329-eb1bfa52a81c admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:52.225908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903dcb3c-60b9-4b64-9329-eb1bfa52a81c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:52.228726 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903dcb3c-60b9-4b64-9329-eb1bfa52a81c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:52.308473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 828/1666] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:43:51 2022] POST /identity/v3/projects => generated 379 bytes in 864 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:52.312998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743630a-a9b3-45db-a891-e53152a4eba2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:52.314649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e743630a-a9b3-45db-a891-e53152a4eba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:52.340975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743630a-a9b3-45db-a891-e53152a4eba2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:52.421166 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:52.521898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bca660-e42c-4ef9-b633-6e4553db3cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:52.522565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bca660-e42c-4ef9-b633-6e4553db3cbc admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:52.522846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bca660-e42c-4ef9-b633-6e4553db3cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:52.523125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bca660-e42c-4ef9-b633-6e4553db3cbc admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:52.527066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bca660-e42c-4ef9-b633-6e4553db3cbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:52.532807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bca660-e42c-4ef9-b633-6e4553db3cbc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:52.632094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 839/1667] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:43:52 2022] POST /identity/v3/users => generated 296 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:52.636273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903dcb3c-60b9-4b64-9329-eb1bfa52a81c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:52.656860 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903dcb3c-60b9-4b64-9329-eb1bfa52a81c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:52.682296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd254a-e721-4a4a-b31e-47b9fa5fc162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:52.682621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd254a-e721-4a4a-b31e-47b9fa5fc162 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:52.683684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd254a-e721-4a4a-b31e-47b9fa5fc162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:52.684238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd254a-e721-4a4a-b31e-47b9fa5fc162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:52.686070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd254a-e721-4a4a-b31e-47b9fa5fc162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:52.689612 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd254a-e721-4a4a-b31e-47b9fa5fc162 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:52.747102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 829/1668] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:43:52.751326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bca660-e42c-4ef9-b633-6e4553db3cbc admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:52.767447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bca660-e42c-4ef9-b633-6e4553db3cbc admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:52.787068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fdb9b-02c5-4db4-be2a-ee5f2046310e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:52.787792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fdb9b-02c5-4db4-be2a-ee5f2046310e admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:52.788093 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fdb9b-02c5-4db4-be2a-ee5f2046310e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:52.788374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fdb9b-02c5-4db4-be2a-ee5f2046310e admin admin] PATH_INFO: `/v3/projects/3b0e004676b447ac869f131e08fa8715/users/fdc4f69a8e034f3eb3cce74c0ab4cdd8/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:52.845804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fdb9b-02c5-4db4-be2a-ee5f2046310e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0e004676b447ac869f131e08fa8715, user_id=fdc4f69a8e034f3eb3cce74c0ab4cdd8, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:52.848885 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fdb9b-02c5-4db4-be2a-ee5f2046310e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:52.886148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 840/1669] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:52 2022] PUT /identity/v3/projects/3b0e004676b447ac869f131e08fa8715/users/fdc4f69a8e034f3eb3cce74c0ab4cdd8/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:52.889174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd254a-e721-4a4a-b31e-47b9fa5fc162 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:52.940308 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd254a-e721-4a4a-b31e-47b9fa5fc162 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:53.070041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81217e0b-f39f-41a6-83ab-8a68f9f52f0c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:53.070559 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81217e0b-f39f-41a6-83ab-8a68f9f52f0c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:53.071043 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81217e0b-f39f-41a6-83ab-8a68f9f52f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:53.209294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-81217e0b-f39f-41a6-83ab-8a68f9f52f0c None None] MFA Rules not processed for user `956aff831fa04d29ba5cffb88e73f4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:53.211488 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-81217e0b-f39f-41a6-83ab-8a68f9f52f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:53.233832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a7bb3-f7f9-466f-9756-172016458fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:53.234869 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a7bb3-f7f9-466f-9756-172016458fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:53.237874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a7bb3-f7f9-466f-9756-172016458fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:53.239107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a7bb3-f7f9-466f-9756-172016458fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:53.245323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a7bb3-f7f9-466f-9756-172016458fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:53.257142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a7bb3-f7f9-466f-9756-172016458fb6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:53.305240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 830/1670] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:53.312634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a7bb3-f7f9-466f-9756-172016458fb6 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:53.361567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a7bb3-f7f9-466f-9756-172016458fb6 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:53.395446 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825475af-b96d-4371-94fe-762606777788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:53.396626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825475af-b96d-4371-94fe-762606777788 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:53.397113 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825475af-b96d-4371-94fe-762606777788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:53.397683 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825475af-b96d-4371-94fe-762606777788 admin admin] PATH_INFO: `/v3/projects/3b0e004676b447ac869f131e08fa8715/users/fdc4f69a8e034f3eb3cce74c0ab4cdd8/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:53.405289 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825475af-b96d-4371-94fe-762606777788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0e004676b447ac869f131e08fa8715, user_id=fdc4f69a8e034f3eb3cce74c0ab4cdd8, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:53.414684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825475af-b96d-4371-94fe-762606777788 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:53.451240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 831/1671] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:53 2022] PUT /identity/v3/projects/3b0e004676b447ac869f131e08fa8715/users/fdc4f69a8e034f3eb3cce74c0ab4cdd8/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:43:53.472791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 841/1672] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:53.479754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7626dcf-05b9-4394-9952-bb6581369ef1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:53.480719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7626dcf-05b9-4394-9952-bb6581369ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:53.482677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7626dcf-05b9-4394-9952-bb6581369ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:53.516119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81217e0b-f39f-41a6-83ab-8a68f9f52f0c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:53.520207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-81217e0b-f39f-41a6-83ab-8a68f9f52f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:53.565116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:53.706441 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b7626dcf-05b9-4394-9952-bb6581369ef1 None None] MFA Rules not processed for user `fdc4f69a8e034f3eb3cce74c0ab4cdd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:53.719842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b7626dcf-05b9-4394-9952-bb6581369ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:54.025525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81217e0b-f39f-41a6-83ab-8a68f9f52f0c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:54.108120 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 832/1673] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 623 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:54.108120 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7626dcf-05b9-4394-9952-bb6581369ef1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:54.111077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b7626dcf-05b9-4394-9952-bb6581369ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:54.131501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:54.414427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27111d07-1e7b-47cd-99ff-42fcf8af17a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:54.416616 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27111d07-1e7b-47cd-99ff-42fcf8af17a2 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:54.417560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27111d07-1e7b-47cd-99ff-42fcf8af17a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:54.418471 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27111d07-1e7b-47cd-99ff-42fcf8af17a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:54.441254 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7626dcf-05b9-4394-9952-bb6581369ef1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:54.462445 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85772e2d-dbca-4f96-ba75-4ecec3f8858d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:54.463345 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85772e2d-dbca-4f96-ba75-4ecec3f8858d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:54.463742 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85772e2d-dbca-4f96-ba75-4ecec3f8858d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:54.464035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85772e2d-dbca-4f96-ba75-4ecec3f8858d admin admin] PATH_INFO: `/v3/projects/c9e156da223844aca1059cd0854e046a` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:54.469742 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85772e2d-dbca-4f96-ba75-4ecec3f8858d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9e156da223844aca1059cd0854e046a)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:54.476045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: WARNING py.warnings [None req-85772e2d-dbca-4f96-ba75-4ecec3f8858d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 10:43:54.476045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: warnings.warn(msg) Aug 03 10:43:54.476045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: Aug 03 10:43:54.478402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85772e2d-dbca-4f96-ba75-4ecec3f8858d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:54.568292 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27111d07-1e7b-47cd-99ff-42fcf8af17a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:54.572388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27111d07-1e7b-47cd-99ff-42fcf8af17a2 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:54.585350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 842/1674] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 1071 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:43:54.600825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27111d07-1e7b-47cd-99ff-42fcf8af17a2 service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:54.604156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-27111d07-1e7b-47cd-99ff-42fcf8af17a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:54.705642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27111d07-1e7b-47cd-99ff-42fcf8af17a2 service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:54.751955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-85772e2d-dbca-4f96-ba75-4ecec3f8858d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9e156da223844aca1059cd0854e046a', 'request_id': 'req-85772e2d-dbca-4f96-ba75-4ecec3f8858d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:54.791250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-85772e2d-dbca-4f96-ba75-4ecec3f8858d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9e156da223844aca1059cd0854e046a', 'request_id': 'req-85772e2d-dbca-4f96-ba75-4ecec3f8858d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:54.810853 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-85772e2d-dbca-4f96-ba75-4ecec3f8858d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9e156da223844aca1059cd0854e046a', 'request_id': 'req-85772e2d-dbca-4f96-ba75-4ecec3f8858d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:54.830138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d1a03-f770-4251-86b7-aa07b4410457 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:54.833045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d1a03-f770-4251-86b7-aa07b4410457 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:54.834700 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d1a03-f770-4251-86b7-aa07b4410457 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:54.835975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d1a03-f770-4251-86b7-aa07b4410457 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:54.842138 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-85772e2d-dbca-4f96-ba75-4ecec3f8858d admin admin] The token cache is being invalidate because project c9e156da223844aca1059cd0854e046a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:43:54.845658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-85772e2d-dbca-4f96-ba75-4ecec3f8858d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:54.849785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-85772e2d-dbca-4f96-ba75-4ecec3f8858d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:43:54.852831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 833/1675] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:43:54 2022] DELETE /identity/v3/projects/c9e156da223844aca1059cd0854e046a => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:54.858428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9df95d-fcd0-445a-b471-3f69efee08cb None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:54.859060 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9df95d-fcd0-445a-b471-3f69efee08cb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:54.859723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9df95d-fcd0-445a-b471-3f69efee08cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:54.911973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5e9df95d-fcd0-445a-b471-3f69efee08cb None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:54.916385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5e9df95d-fcd0-445a-b471-3f69efee08cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:54.926672 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:54.966270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:55.013299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d1a03-f770-4251-86b7-aa07b4410457 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:55.015638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d1a03-f770-4251-86b7-aa07b4410457 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:55.042454 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 843/1676] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:43:55.154570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 834/1677] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:43:55.171650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2a355-2c7b-475b-a8ea-33d7e459aed7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:55.172278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2a355-2c7b-475b-a8ea-33d7e459aed7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:55.172511 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2a355-2c7b-475b-a8ea-33d7e459aed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:55.208422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9df95d-fcd0-445a-b471-3f69efee08cb None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:55.211857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5e9df95d-fcd0-445a-b471-3f69efee08cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:55.279113 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9df95d-fcd0-445a-b471-3f69efee08cb None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:55.282216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d1e2a355-2c7b-475b-a8ea-33d7e459aed7 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:55.284470 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d1e2a355-2c7b-475b-a8ea-33d7e459aed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:55.294820 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208fd61-be17-4a46-baa6-bcb631a6cccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:55.296183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208fd61-be17-4a46-baa6-bcb631a6cccb service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:55.296681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208fd61-be17-4a46-baa6-bcb631a6cccb service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:55.297183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208fd61-be17-4a46-baa6-bcb631a6cccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:55.301542 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a208fd61-be17-4a46-baa6-bcb631a6cccb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:55.324048 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 844/1678] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:55.331246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2a355-2c7b-475b-a8ea-33d7e459aed7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:55.354261 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2a355-2c7b-475b-a8ea-33d7e459aed7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:55.373163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325e5cb-953f-4596-b27f-9bdafba3ff78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:55.374429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325e5cb-953f-4596-b27f-9bdafba3ff78 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:55.374429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325e5cb-953f-4596-b27f-9bdafba3ff78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:55.374429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325e5cb-953f-4596-b27f-9bdafba3ff78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:55.375099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325e5cb-953f-4596-b27f-9bdafba3ff78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:43:55.380599 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325e5cb-953f-4596-b27f-9bdafba3ff78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:55.382450 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325e5cb-953f-4596-b27f-9bdafba3ff78 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:55.429196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 845/1679] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:43:55.437640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325e5cb-953f-4596-b27f-9bdafba3ff78 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:55.479297 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325e5cb-953f-4596-b27f-9bdafba3ff78 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:55.497650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e73321-fbbc-4e9f-804d-5f7be6280f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:55.498300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e73321-fbbc-4e9f-804d-5f7be6280f3a admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:55.498756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e73321-fbbc-4e9f-804d-5f7be6280f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:55.499068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e73321-fbbc-4e9f-804d-5f7be6280f3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:55.499922 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e73321-fbbc-4e9f-804d-5f7be6280f3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:55.502963 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208fd61-be17-4a46-baa6-bcb631a6cccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:55.505497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208fd61-be17-4a46-baa6-bcb631a6cccb service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:55.510005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e73321-fbbc-4e9f-804d-5f7be6280f3a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:55.539140 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 835/1680] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:43:55.603563 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 846/1681] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:43:55 2022] POST /identity/v3/projects => generated 389 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:55.612153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208fd61-be17-4a46-baa6-bcb631a6cccb service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:55.670986 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208fd61-be17-4a46-baa6-bcb631a6cccb service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:55.951945 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd418090-5812-4073-812f-c3d728c60447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:55.952908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd418090-5812-4073-812f-c3d728c60447 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:55.953459 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd418090-5812-4073-812f-c3d728c60447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:55.953861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd418090-5812-4073-812f-c3d728c60447 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:55.956029 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd418090-5812-4073-812f-c3d728c60447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:55.964662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd418090-5812-4073-812f-c3d728c60447 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:56.020616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 836/1682] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:43:55 2022] POST /identity/v3/users => generated 295 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:56.024350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e73321-fbbc-4e9f-804d-5f7be6280f3a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:56.063161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e73321-fbbc-4e9f-804d-5f7be6280f3a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:56.077537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597866e-a442-4545-b1dd-b632b2b62eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:56.080026 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597866e-a442-4545-b1dd-b632b2b62eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:56.080813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597866e-a442-4545-b1dd-b632b2b62eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:56.081667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597866e-a442-4545-b1dd-b632b2b62eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:56.084715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597866e-a442-4545-b1dd-b632b2b62eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:56.090748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597866e-a442-4545-b1dd-b632b2b62eb3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:56.131431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 847/1683] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:56.132166 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd418090-5812-4073-812f-c3d728c60447 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:56.169847 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd418090-5812-4073-812f-c3d728c60447 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:56.209269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d63595-db79-4b7e-88ef-7fc72c7991a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:56.216250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d63595-db79-4b7e-88ef-7fc72c7991a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:56.216848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d63595-db79-4b7e-88ef-7fc72c7991a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:56.217375 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d63595-db79-4b7e-88ef-7fc72c7991a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:56.219493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d63595-db79-4b7e-88ef-7fc72c7991a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:56.236744 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d63595-db79-4b7e-88ef-7fc72c7991a4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:56.315341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597866e-a442-4545-b1dd-b632b2b62eb3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:56.324743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 837/1684] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:56.384174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597866e-a442-4545-b1dd-b632b2b62eb3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:56.410051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd317c-3b4d-41d8-9482-80d25e455101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:56.413018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd317c-3b4d-41d8-9482-80d25e455101 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:56.418232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd317c-3b4d-41d8-9482-80d25e455101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:56.418942 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd317c-3b4d-41d8-9482-80d25e455101 admin admin] PATH_INFO: `/v3/projects/ef92326538fc4b68a51d275e93a2ba43/users/ca058d3400d64055ad78dfc5f631aa46/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:56.520688 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd317c-3b4d-41d8-9482-80d25e455101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef92326538fc4b68a51d275e93a2ba43, user_id=ca058d3400d64055ad78dfc5f631aa46, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:56.537430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd317c-3b4d-41d8-9482-80d25e455101 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:56.596778 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 848/1685] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:56 2022] PUT /identity/v3/projects/ef92326538fc4b68a51d275e93a2ba43/users/ca058d3400d64055ad78dfc5f631aa46/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:56.622884 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f8611-55c4-4b15-89d3-6abb121fc0a0 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:56.623381 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f8611-55c4-4b15-89d3-6abb121fc0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:56.626315 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f8611-55c4-4b15-89d3-6abb121fc0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:56.777718 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-039f8611-55c4-4b15-89d3-6abb121fc0a0 None None] MFA Rules not processed for user `ca058d3400d64055ad78dfc5f631aa46`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:56.779952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-039f8611-55c4-4b15-89d3-6abb121fc0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:56.872982 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:56.952468 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 838/1686] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:56.959240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd317c-3b4d-41d8-9482-80d25e455101 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:57.124134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd317c-3b4d-41d8-9482-80d25e455101 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:57.137059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce1a2c-f239-43c1-9c16-6134a0f45773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:57.137059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce1a2c-f239-43c1-9c16-6134a0f45773 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:57.137059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce1a2c-f239-43c1-9c16-6134a0f45773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:57.140721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce1a2c-f239-43c1-9c16-6134a0f45773 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:57.141771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce1a2c-f239-43c1-9c16-6134a0f45773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:57.144022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce1a2c-f239-43c1-9c16-6134a0f45773 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:57.213883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 849/1687] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:43:56 2022] POST /identity/v3/projects => generated 387 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:57.218907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f8611-55c4-4b15-89d3-6abb121fc0a0 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:57.248144 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f8611-55c4-4b15-89d3-6abb121fc0a0 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:57.437346 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8dbcd-95fa-4c7c-b40e-43f11482cd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:57.440621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8dbcd-95fa-4c7c-b40e-43f11482cd38 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:57.442057 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8dbcd-95fa-4c7c-b40e-43f11482cd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:57.442402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8dbcd-95fa-4c7c-b40e-43f11482cd38 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:57.448055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8dbcd-95fa-4c7c-b40e-43f11482cd38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:57.451384 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8dbcd-95fa-4c7c-b40e-43f11482cd38 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:57.552790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 839/1688] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:43:57 2022] POST /identity/v3/users => generated 300 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:43:57.558488 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce1a2c-f239-43c1-9c16-6134a0f45773 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:57.572521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce1a2c-f239-43c1-9c16-6134a0f45773 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:57.605614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bc0f1-39ef-4632-b409-126de7c93755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:57.626172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bc0f1-39ef-4632-b409-126de7c93755 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:57.626856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bc0f1-39ef-4632-b409-126de7c93755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:57.627272 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bc0f1-39ef-4632-b409-126de7c93755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:57.628565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175bc0f1-39ef-4632-b409-126de7c93755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:57.644094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175bc0f1-39ef-4632-b409-126de7c93755 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:57.700184 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 850/1689] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:57.708813 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8dbcd-95fa-4c7c-b40e-43f11482cd38 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:57.742952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8dbcd-95fa-4c7c-b40e-43f11482cd38 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:57.765318 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752b9e7-0000-4fb6-a132-e4cc12c2fda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:57.768598 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752b9e7-0000-4fb6-a132-e4cc12c2fda7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:57.771489 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752b9e7-0000-4fb6-a132-e4cc12c2fda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:57.772238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752b9e7-0000-4fb6-a132-e4cc12c2fda7 admin admin] PATH_INFO: `/v3/projects/d8fcc6ac21b34f6eb815281956b4b828/users/78a0a71b41974b31b7efb6ac088d658a/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:57.857840 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752b9e7-0000-4fb6-a132-e4cc12c2fda7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8fcc6ac21b34f6eb815281956b4b828, user_id=78a0a71b41974b31b7efb6ac088d658a, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:57.874217 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752b9e7-0000-4fb6-a132-e4cc12c2fda7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:57.924279 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 840/1690] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:57 2022] PUT /identity/v3/projects/d8fcc6ac21b34f6eb815281956b4b828/users/78a0a71b41974b31b7efb6ac088d658a/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:43:57.930288 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bc0f1-39ef-4632-b409-126de7c93755 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:57.944671 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bc0f1-39ef-4632-b409-126de7c93755 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:58.011771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:43:58.147138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0d3a9-84df-44a3-81c0-696cf4c71b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:58.150661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0d3a9-84df-44a3-81c0-696cf4c71b73 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:58.151185 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0d3a9-84df-44a3-81c0-696cf4c71b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:58.151678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0d3a9-84df-44a3-81c0-696cf4c71b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:58.157482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0d3a9-84df-44a3-81c0-696cf4c71b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:58.167881 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0d3a9-84df-44a3-81c0-696cf4c71b73 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:58.214797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 851/1691] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:43:58.221851 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752b9e7-0000-4fb6-a132-e4cc12c2fda7 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:58.284366 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752b9e7-0000-4fb6-a132-e4cc12c2fda7 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:58.341773 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f159d-1b32-4542-b82c-8b98b60046db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:58.346442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f159d-1b32-4542-b82c-8b98b60046db admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:58.352429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f159d-1b32-4542-b82c-8b98b60046db admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:58.358467 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f159d-1b32-4542-b82c-8b98b60046db admin admin] PATH_INFO: `/v3/projects/d8fcc6ac21b34f6eb815281956b4b828/users/78a0a71b41974b31b7efb6ac088d658a/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:58.375492 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f159d-1b32-4542-b82c-8b98b60046db admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8fcc6ac21b34f6eb815281956b4b828, user_id=78a0a71b41974b31b7efb6ac088d658a, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:58.380694 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f159d-1b32-4542-b82c-8b98b60046db admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:58.416343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 841/1692] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:43:58 2022] PUT /identity/v3/projects/d8fcc6ac21b34f6eb815281956b4b828/users/78a0a71b41974b31b7efb6ac088d658a/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:43:58.419014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcacd4-f7a4-427d-b027-98966d9eb76d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:58.419503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcacd4-f7a4-427d-b027-98966d9eb76d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:58.419991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcacd4-f7a4-427d-b027-98966d9eb76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:58.527924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c5dcacd4-f7a4-427d-b027-98966d9eb76d None None] MFA Rules not processed for user `78a0a71b41974b31b7efb6ac088d658a`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:43:58.530183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c5dcacd4-f7a4-427d-b027-98966d9eb76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:58.768060 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 852/1693] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:43:58 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:43:58.773515 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f159d-1b32-4542-b82c-8b98b60046db admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:58.776444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-042f159d-1b32-4542-b82c-8b98b60046db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:43:58.956872 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f159d-1b32-4542-b82c-8b98b60046db admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:58.972982 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4f67e-b48f-4385-8674-bc31ce2684a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:58.978966 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4f67e-b48f-4385-8674-bc31ce2684a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:58.980261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4f67e-b48f-4385-8674-bc31ce2684a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:58.980555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4f67e-b48f-4385-8674-bc31ce2684a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:59.039691 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4f67e-b48f-4385-8674-bc31ce2684a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:59.043005 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4f67e-b48f-4385-8674-bc31ce2684a9 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:43:59.086349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 842/1694] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:43:58 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:43:59.401603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcacd4-f7a4-427d-b027-98966d9eb76d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:43:59.697816 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcacd4-f7a4-427d-b027-98966d9eb76d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:43:59.740180 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cef46-7cf3-42f9-a115-eb0da517c9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:43:59.740387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cef46-7cf3-42f9-a115-eb0da517c9e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:43:59.741783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cef46-7cf3-42f9-a115-eb0da517c9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:43:59.742376 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cef46-7cf3-42f9-a115-eb0da517c9e6 admin admin] PATH_INFO: `/v3/users/ca058d3400d64055ad78dfc5f631aa46` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:43:59.745018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cef46-7cf3-42f9-a115-eb0da517c9e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca058d3400d64055ad78dfc5f631aa46)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:43:59.773507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cef46-7cf3-42f9-a115-eb0da517c9e6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:00.038702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:00.043228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6d6cef46-7cf3-42f9-a115-eb0da517c9e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca058d3400d64055ad78dfc5f631aa46'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:00.087088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536075a3-4684-4bed-8cdb-634e2e48e5fe None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:00.088001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536075a3-4684-4bed-8cdb-634e2e48e5fe None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:00.088596 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536075a3-4684-4bed-8cdb-634e2e48e5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:00.100039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6d6cef46-7cf3-42f9-a115-eb0da517c9e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca058d3400d64055ad78dfc5f631aa46'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:00.112297 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6d6cef46-7cf3-42f9-a115-eb0da517c9e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca058d3400d64055ad78dfc5f631aa46'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:00.148121 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6d6cef46-7cf3-42f9-a115-eb0da517c9e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca058d3400d64055ad78dfc5f631aa46'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:00.163472 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6d6cef46-7cf3-42f9-a115-eb0da517c9e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca058d3400d64055ad78dfc5f631aa46'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:00.166433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-536075a3-4684-4bed-8cdb-634e2e48e5fe None None] MFA Rules not processed for user `ebdb81686d804eb3864b97481643fc2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:00.170679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-536075a3-4684-4bed-8cdb-634e2e48e5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:00.183164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 853/1695] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:43:59 2022] DELETE /identity/v3/users/ca058d3400d64055ad78dfc5f631aa46 => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:00.188585 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7796fa-4b37-4fd7-b29b-b47c2caf2189 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:00.189230 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7796fa-4b37-4fd7-b29b-b47c2caf2189 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:00.190280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7796fa-4b37-4fd7-b29b-b47c2caf2189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:00.245828 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6c7796fa-4b37-4fd7-b29b-b47c2caf2189 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:00.247653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6c7796fa-4b37-4fd7-b29b-b47c2caf2189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:00.250659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:00.367009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 843/1696] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:00.424763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 854/1697] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:00.433937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982703e-a1c5-4179-bebb-f49130122780 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:00.434425 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982703e-a1c5-4179-bebb-f49130122780 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:00.434805 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982703e-a1c5-4179-bebb-f49130122780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:00.465521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7796fa-4b37-4fd7-b29b-b47c2caf2189 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:00.470129 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6c7796fa-4b37-4fd7-b29b-b47c2caf2189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:00.501128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-1982703e-a1c5-4179-bebb-f49130122780 None None] MFA Rules not processed for user `956aff831fa04d29ba5cffb88e73f4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:00.505438 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1982703e-a1c5-4179-bebb-f49130122780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:00.612602 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7796fa-4b37-4fd7-b29b-b47c2caf2189 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:00.618431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52205e-805e-4687-b487-be1dd41c7bfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:00.621537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52205e-805e-4687-b487-be1dd41c7bfc service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:00.622160 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52205e-805e-4687-b487-be1dd41c7bfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:00.622682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52205e-805e-4687-b487-be1dd41c7bfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:00.654635 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52205e-805e-4687-b487-be1dd41c7bfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:00.656722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52205e-805e-4687-b487-be1dd41c7bfc service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:00.710778 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 855/1698] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:44:00.718033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 844/1699] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:01.489204 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed216661-bf25-4750-83f6-85190999863f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:01.489817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed216661-bf25-4750-83f6-85190999863f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:01.489971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed216661-bf25-4750-83f6-85190999863f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:01.560776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ed216661-bf25-4750-83f6-85190999863f None None] MFA Rules not processed for user `d560ec3b091c4baab93c4abb5fabd287`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:01.563651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ed216661-bf25-4750-83f6-85190999863f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:01.684609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 845/1700] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:01.710429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5c93b-007d-4c2c-b0a4-4f40dd2345c3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:01.711217 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5c93b-007d-4c2c-b0a4-4f40dd2345c3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:01.711217 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5c93b-007d-4c2c-b0a4-4f40dd2345c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:01.763607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-45f5c93b-007d-4c2c-b0a4-4f40dd2345c3 None None] MFA Rules not processed for user `d560ec3b091c4baab93c4abb5fabd287`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:01.765112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-45f5c93b-007d-4c2c-b0a4-4f40dd2345c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:01.784885 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 856/1701] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:01.914205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed216661-bf25-4750-83f6-85190999863f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:01.916914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ed216661-bf25-4750-83f6-85190999863f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:01.990139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed216661-bf25-4750-83f6-85190999863f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:02.011147 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005bb3a5-693d-4d95-99fe-3058a4120e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:02.012601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bb3a5-693d-4d95-99fe-3058a4120e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:02.012970 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bb3a5-693d-4d95-99fe-3058a4120e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:02.013609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bb3a5-693d-4d95-99fe-3058a4120e4f admin admin] PATH_INFO: `/v3/projects/ef92326538fc4b68a51d275e93a2ba43` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:02.020284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005bb3a5-693d-4d95-99fe-3058a4120e4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef92326538fc4b68a51d275e93a2ba43)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:02.024676 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005bb3a5-693d-4d95-99fe-3058a4120e4f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:02.201830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-005bb3a5-693d-4d95-99fe-3058a4120e4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef92326538fc4b68a51d275e93a2ba43', 'request_id': 'req-005bb3a5-693d-4d95-99fe-3058a4120e4f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:02.250719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-005bb3a5-693d-4d95-99fe-3058a4120e4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef92326538fc4b68a51d275e93a2ba43', 'request_id': 'req-005bb3a5-693d-4d95-99fe-3058a4120e4f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:02.270269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:02.283999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-005bb3a5-693d-4d95-99fe-3058a4120e4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef92326538fc4b68a51d275e93a2ba43', 'request_id': 'req-005bb3a5-693d-4d95-99fe-3058a4120e4f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:02.321059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-005bb3a5-693d-4d95-99fe-3058a4120e4f admin admin] The token cache is being invalidate because project ef92326538fc4b68a51d275e93a2ba43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:02.322505 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-005bb3a5-693d-4d95-99fe-3058a4120e4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:02.325267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-005bb3a5-693d-4d95-99fe-3058a4120e4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:02.329730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 846/1702] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:01 2022] DELETE /identity/v3/projects/ef92326538fc4b68a51d275e93a2ba43 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:02.335888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5c93b-007d-4c2c-b0a4-4f40dd2345c3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:02.370756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5c93b-007d-4c2c-b0a4-4f40dd2345c3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:02.508127 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a3877-fd36-4c9d-81bf-12a8d34aa0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:02.508793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a3877-fd36-4c9d-81bf-12a8d34aa0d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:02.509119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a3877-fd36-4c9d-81bf-12a8d34aa0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:02.509406 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a3877-fd36-4c9d-81bf-12a8d34aa0d4 admin admin] PATH_INFO: `/v3/users/78a0a71b41974b31b7efb6ac088d658a` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:02.511864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a3877-fd36-4c9d-81bf-12a8d34aa0d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a0a71b41974b31b7efb6ac088d658a)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:02.513918 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a3877-fd36-4c9d-81bf-12a8d34aa0d4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:02.516369 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:02.597233 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6b8a3877-fd36-4c9d-81bf-12a8d34aa0d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a0a71b41974b31b7efb6ac088d658a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:02.620966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6b8a3877-fd36-4c9d-81bf-12a8d34aa0d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a0a71b41974b31b7efb6ac088d658a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:02.624722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6b8a3877-fd36-4c9d-81bf-12a8d34aa0d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a0a71b41974b31b7efb6ac088d658a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:02.640482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6b8a3877-fd36-4c9d-81bf-12a8d34aa0d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a0a71b41974b31b7efb6ac088d658a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:02.651952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6b8a3877-fd36-4c9d-81bf-12a8d34aa0d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a0a71b41974b31b7efb6ac088d658a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:02.661920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 857/1703] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:02 2022] DELETE /identity/v3/users/78a0a71b41974b31b7efb6ac088d658a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:03.362235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71e468-e45f-4fc5-b2fa-6002e667856e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:03.362814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71e468-e45f-4fc5-b2fa-6002e667856e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:03.364096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71e468-e45f-4fc5-b2fa-6002e667856e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:03.471477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a3877-fd36-4c9d-81bf-12a8d34aa0d4 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:03.473250 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6b8a3877-fd36-4c9d-81bf-12a8d34aa0d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:03.519768 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4d71e468-e45f-4fc5-b2fa-6002e667856e None None] MFA Rules not processed for user `5744cccc5a224263b71c052dc783e103`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:03.530887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4d71e468-e45f-4fc5-b2fa-6002e667856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:03.742225 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a3877-fd36-4c9d-81bf-12a8d34aa0d4 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:03.743160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 847/1704] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:03.746157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0cb0f-609d-4917-98bd-545f28d4d3f7 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:03.746157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0cb0f-609d-4917-98bd-545f28d4d3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:03.746157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0cb0f-609d-4917-98bd-545f28d4d3f7 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:03.746733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 848/1705] 10.222.0.47 () {54 vars in 912 bytes} [Wed Aug 3 10:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:44:03.753049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b22eb7-77b6-46ac-a36d-b9ce27fc0b3d None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:03.753336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b22eb7-77b6-46ac-a36d-b9ce27fc0b3d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:03.753652 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b22eb7-77b6-46ac-a36d-b9ce27fc0b3d None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:03.755737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 849/1706] 10.222.0.47 () {54 vars in 912 bytes} [Wed Aug 3 10:44:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:44:03.757252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd7887-04c4-4281-96db-1ab7aedf36a5 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:03.757739 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd7887-04c4-4281-96db-1ab7aedf36a5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:03.758186 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd7887-04c4-4281-96db-1ab7aedf36a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:03.763982 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410149fc-e83f-4703-a030-8906d798ad33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:03.766560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410149fc-e83f-4703-a030-8906d798ad33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:03.768954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410149fc-e83f-4703-a030-8906d798ad33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:03.769592 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410149fc-e83f-4703-a030-8906d798ad33 admin admin] PATH_INFO: `/v3/projects/d8fcc6ac21b34f6eb815281956b4b828` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:03.775215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410149fc-e83f-4703-a030-8906d798ad33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8fcc6ac21b34f6eb815281956b4b828)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:03.799549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410149fc-e83f-4703-a030-8906d798ad33 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:03.843946 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7fbd7887-04c4-4281-96db-1ab7aedf36a5 None None] MFA Rules not processed for user `1c211cc19a2045bd9383d31c7e81ceac`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:03.845822 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7fbd7887-04c4-4281-96db-1ab7aedf36a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:04.038889 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-410149fc-e83f-4703-a030-8906d798ad33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8fcc6ac21b34f6eb815281956b4b828', 'request_id': 'req-410149fc-e83f-4703-a030-8906d798ad33'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:04.062739 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-410149fc-e83f-4703-a030-8906d798ad33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8fcc6ac21b34f6eb815281956b4b828', 'request_id': 'req-410149fc-e83f-4703-a030-8906d798ad33'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:04.087533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-410149fc-e83f-4703-a030-8906d798ad33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8fcc6ac21b34f6eb815281956b4b828', 'request_id': 'req-410149fc-e83f-4703-a030-8906d798ad33'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:04.126150 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 850/1707] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 3 10:44:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:04.136323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab047648-8f35-4e9b-b3be-314f3e8eb4e3 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:04.137226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab047648-8f35-4e9b-b3be-314f3e8eb4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:04.137226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab047648-8f35-4e9b-b3be-314f3e8eb4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:04.158802 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-410149fc-e83f-4703-a030-8906d798ad33 admin admin] The token cache is being invalidate because project d8fcc6ac21b34f6eb815281956b4b828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:04.159323 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-410149fc-e83f-4703-a030-8906d798ad33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:04.180514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-410149fc-e83f-4703-a030-8906d798ad33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:04.199776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 858/1708] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:03 2022] DELETE /identity/v3/projects/d8fcc6ac21b34f6eb815281956b4b828 => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:44:04.202444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410149fc-e83f-4703-a030-8906d798ad33 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:04.209319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-410149fc-e83f-4703-a030-8906d798ad33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:04.243333 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ab047648-8f35-4e9b-b3be-314f3e8eb4e3 None None] MFA Rules not processed for user `1c211cc19a2045bd9383d31c7e81ceac`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:04.253222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ab047648-8f35-4e9b-b3be-314f3e8eb4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:04.293670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:04.453436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 851/1709] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 3 10:44:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:04.455139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a30c00-a64d-4015-80f5-bc89a83b8e10 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:04.455645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a30c00-a64d-4015-80f5-bc89a83b8e10 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:04.455954 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a30c00-a64d-4015-80f5-bc89a83b8e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:04.500393 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d3a30c00-a64d-4015-80f5-bc89a83b8e10 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:04.501783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d3a30c00-a64d-4015-80f5-bc89a83b8e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:04.539748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410149fc-e83f-4703-a030-8906d798ad33 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:04.559530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c617010-d554-4661-b21f-d99c790ad242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:04.561015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c617010-d554-4661-b21f-d99c790ad242 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:04.561821 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c617010-d554-4661-b21f-d99c790ad242 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:04.562508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c617010-d554-4661-b21f-d99c790ad242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:04.568039 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:04.575396 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9c617010-d554-4661-b21f-d99c790ad242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:04.620794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c617010-d554-4661-b21f-d99c790ad242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:04.622547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c617010-d554-4661-b21f-d99c790ad242 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:04.655682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 859/1710] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:04.663889 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c617010-d554-4661-b21f-d99c790ad242 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:04.699631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c617010-d554-4661-b21f-d99c790ad242 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:04.709108 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be24592-06b2-440e-87a5-9c4543ca7ec7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:04.710529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be24592-06b2-440e-87a5-9c4543ca7ec7 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:04.718716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be24592-06b2-440e-87a5-9c4543ca7ec7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:04.718930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be24592-06b2-440e-87a5-9c4543ca7ec7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:04.731086 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 852/1711] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:04.745641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be24592-06b2-440e-87a5-9c4543ca7ec7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:04.748446 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c429c-bd91-4473-8e1f-c406a873001d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:04.748732 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c429c-bd91-4473-8e1f-c406a873001d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:04.749175 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c429c-bd91-4473-8e1f-c406a873001d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:04.752619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be24592-06b2-440e-87a5-9c4543ca7ec7 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:04.791059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 860/1712] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:04.818185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-9c8c429c-bd91-4473-8e1f-c406a873001d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:04.819473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9c8c429c-bd91-4473-8e1f-c406a873001d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:04.837604 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 853/1713] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:04.844328 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be24592-06b2-440e-87a5-9c4543ca7ec7 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:04.878866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be24592-06b2-440e-87a5-9c4543ca7ec7 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:04.901811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d7c34-da2a-454d-a734-77bae6213d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:04.902462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d7c34-da2a-454d-a734-77bae6213d78 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:04.902746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d7c34-da2a-454d-a734-77bae6213d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:04.903019 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d7c34-da2a-454d-a734-77bae6213d78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:04.903747 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d7c34-da2a-454d-a734-77bae6213d78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:44:04.904054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d7c34-da2a-454d-a734-77bae6213d78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:04.905755 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d7c34-da2a-454d-a734-77bae6213d78 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:04.946920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 861/1714] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:44:04.953727 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c429c-bd91-4473-8e1f-c406a873001d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:04.974024 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c429c-bd91-4473-8e1f-c406a873001d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:04.983417 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a414612-2a65-45f1-9d77-463acaa3d0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:04.984072 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a414612-2a65-45f1-9d77-463acaa3d0a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:04.984367 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a414612-2a65-45f1-9d77-463acaa3d0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:04.984776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a414612-2a65-45f1-9d77-463acaa3d0a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:04.985550 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a414612-2a65-45f1-9d77-463acaa3d0a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:04.987712 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a414612-2a65-45f1-9d77-463acaa3d0a7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:05.110616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 854/1715] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:04 2022] POST /identity/v3/projects => generated 379 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:44:05.112749 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d7c34-da2a-454d-a734-77bae6213d78 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:05.137318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d7c34-da2a-454d-a734-77bae6213d78 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:05.354175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce1138-3554-4fed-9528-5e13f88c5c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:05.356310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce1138-3554-4fed-9528-5e13f88c5c41 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:05.360420 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce1138-3554-4fed-9528-5e13f88c5c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:05.361123 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce1138-3554-4fed-9528-5e13f88c5c41 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:05.364763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce1138-3554-4fed-9528-5e13f88c5c41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:05.379722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce1138-3554-4fed-9528-5e13f88c5c41 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:05.449781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 862/1716] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:05 2022] POST /identity/v3/users => generated 290 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:05.456526 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a414612-2a65-45f1-9d77-463acaa3d0a7 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:05.486048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a414612-2a65-45f1-9d77-463acaa3d0a7 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:05.524491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c38966-4ec7-4092-a0b2-079aa0ab03c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:05.524491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c38966-4ec7-4092-a0b2-079aa0ab03c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:05.524491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c38966-4ec7-4092-a0b2-079aa0ab03c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:05.524491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c38966-4ec7-4092-a0b2-079aa0ab03c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:05.525110 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c38966-4ec7-4092-a0b2-079aa0ab03c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:05.525302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c38966-4ec7-4092-a0b2-079aa0ab03c1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:05.554693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 855/1717] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:05.564549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce1138-3554-4fed-9528-5e13f88c5c41 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:05.583354 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce1138-3554-4fed-9528-5e13f88c5c41 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:05.597246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb5dbf-3d32-49a6-a9a0-8d246f15c94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:05.601139 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb5dbf-3d32-49a6-a9a0-8d246f15c94d admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:05.602614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb5dbf-3d32-49a6-a9a0-8d246f15c94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:05.617078 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb5dbf-3d32-49a6-a9a0-8d246f15c94d admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:05.620370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb5dbf-3d32-49a6-a9a0-8d246f15c94d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:05.662863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb5dbf-3d32-49a6-a9a0-8d246f15c94d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:05.746063 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 863/1718] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:44:05.747611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c38966-4ec7-4092-a0b2-079aa0ab03c1 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:05.769256 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c38966-4ec7-4092-a0b2-079aa0ab03c1 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:05.788658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3ea4c-1835-4a4d-966e-6df5ac0bc054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:05.790059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3ea4c-1835-4a4d-966e-6df5ac0bc054 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:05.790581 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3ea4c-1835-4a4d-966e-6df5ac0bc054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:05.791041 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3ea4c-1835-4a4d-966e-6df5ac0bc054 admin admin] PATH_INFO: `/v3/projects/d03ed41586994666b3b447d1ee316ef8/users/7c261a283dc74e2f8b10a4d1780294b2/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:05.827120 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3ea4c-1835-4a4d-966e-6df5ac0bc054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d03ed41586994666b3b447d1ee316ef8, user_id=7c261a283dc74e2f8b10a4d1780294b2, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:05.831872 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3ea4c-1835-4a4d-966e-6df5ac0bc054 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:05.871347 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 856/1719] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:05 2022] PUT /identity/v3/projects/d03ed41586994666b3b447d1ee316ef8/users/7c261a283dc74e2f8b10a4d1780294b2/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:05.899365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28423e50-0eb6-4853-8f00-eb4a199da065 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:05.903183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28423e50-0eb6-4853-8f00-eb4a199da065 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:05.904125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28423e50-0eb6-4853-8f00-eb4a199da065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:06.051878 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-28423e50-0eb6-4853-8f00-eb4a199da065 None None] MFA Rules not processed for user `7c261a283dc74e2f8b10a4d1780294b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:06.054562 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-28423e50-0eb6-4853-8f00-eb4a199da065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:06.189162 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:06.224856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 864/1720] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:06.231767 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3ea4c-1835-4a4d-966e-6df5ac0bc054 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:06.378770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3ea4c-1835-4a4d-966e-6df5ac0bc054 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:06.385521 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28dcb0-a989-444d-ae3f-4bdd9b8d235a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:06.386849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28dcb0-a989-444d-ae3f-4bdd9b8d235a admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:06.387162 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28dcb0-a989-444d-ae3f-4bdd9b8d235a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:06.387436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28dcb0-a989-444d-ae3f-4bdd9b8d235a admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:06.388130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28dcb0-a989-444d-ae3f-4bdd9b8d235a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:06.391064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28dcb0-a989-444d-ae3f-4bdd9b8d235a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:06.445101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 857/1721] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:06 2022] POST /identity/v3/projects => generated 379 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:06.452897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28423e50-0eb6-4853-8f00-eb4a199da065 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:06.479745 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1da1d7-d640-46c5-9612-c1c9230054e6 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:06.480321 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1da1d7-d640-46c5-9612-c1c9230054e6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:06.480405 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1da1d7-d640-46c5-9612-c1c9230054e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:06.511625 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28423e50-0eb6-4853-8f00-eb4a199da065 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:06.556370 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3d1da1d7-d640-46c5-9612-c1c9230054e6 None None] MFA Rules not processed for user `a5e714aad25f4fda8acf767ad7e2490b`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:06.558098 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3d1da1d7-d640-46c5-9612-c1c9230054e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:06.703471 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:06.780266 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 858/1722] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:06.781548 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1da1d7-d640-46c5-9612-c1c9230054e6 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:06.783036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3d1da1d7-d640-46c5-9612-c1c9230054e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:06.883066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725f2af-eae6-43f7-ad58-5ae7c1de14ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:06.884109 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725f2af-eae6-43f7-ad58-5ae7c1de14ca admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:06.884595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725f2af-eae6-43f7-ad58-5ae7c1de14ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:06.885075 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725f2af-eae6-43f7-ad58-5ae7c1de14ca admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:06.886432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725f2af-eae6-43f7-ad58-5ae7c1de14ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:06.892163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725f2af-eae6-43f7-ad58-5ae7c1de14ca admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:07.008384 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 865/1723] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:06 2022] POST /identity/v3/users => generated 296 bytes in 557 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:44:07.011822 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2e636-6414-424c-a711-5ed114753a18 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:07.014502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2e636-6414-424c-a711-5ed114753a18 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:07.015815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2e636-6414-424c-a711-5ed114753a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:07.025971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1da1d7-d640-46c5-9612-c1c9230054e6 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:07.032991 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16b11a-e0a7-441d-8ce8-dc21ca1a5b02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:07.033762 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16b11a-e0a7-441d-8ce8-dc21ca1a5b02 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:07.034059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16b11a-e0a7-441d-8ce8-dc21ca1a5b02 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:07.034356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16b11a-e0a7-441d-8ce8-dc21ca1a5b02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:07.037429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7c16b11a-e0a7-441d-8ce8-dc21ca1a5b02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:07.059337 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c1d2e636-6414-424c-a711-5ed114753a18 None None] MFA Rules not processed for user `a5e714aad25f4fda8acf767ad7e2490b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:07.063129 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c1d2e636-6414-424c-a711-5ed114753a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:07.083784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 866/1724] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:07.086017 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1d709-f0c3-4997-92b3-5f2865b365b4 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:07.086743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1d709-f0c3-4997-92b3-5f2865b365b4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:07.087867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1d709-f0c3-4997-92b3-5f2865b365b4 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:07.089908 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 867/1725] 10.222.0.47 () {54 vars in 961 bytes} [Wed Aug 3 10:44:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:44:07.092779 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1d709-f0c3-4997-92b3-5f2865b365b4 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:07.096280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-44e1d709-f0c3-4997-92b3-5f2865b365b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:07.131750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1d709-f0c3-4997-92b3-5f2865b365b4 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:07.141264 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78992692-8669-4c11-b3a8-9edac935ccfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:07.142387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78992692-8669-4c11-b3a8-9edac935ccfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:07.142863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78992692-8669-4c11-b3a8-9edac935ccfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:07.143314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78992692-8669-4c11-b3a8-9edac935ccfd admin admin] PATH_INFO: `/v3/users/82c9f163a3764a30b7f938b362900b69` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:07.150701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78992692-8669-4c11-b3a8-9edac935ccfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c9f163a3764a30b7f938b362900b69)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:07.154443 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78992692-8669-4c11-b3a8-9edac935ccfd admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:07.180550 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16b11a-e0a7-441d-8ce8-dc21ca1a5b02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:07.183910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16b11a-e0a7-441d-8ce8-dc21ca1a5b02 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:07.201270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 859/1726] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:07.202639 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16b11a-e0a7-441d-8ce8-dc21ca1a5b02 service nova] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:07.220974 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16b11a-e0a7-441d-8ce8-dc21ca1a5b02 service nova] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:07.229372 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c3519-0bae-4735-83e7-9e15d14edc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:07.230292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c3519-0bae-4735-83e7-9e15d14edc92 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:07.230878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c3519-0bae-4735-83e7-9e15d14edc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:07.231323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c3519-0bae-4735-83e7-9e15d14edc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:07.232541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c3519-0bae-4735-83e7-9e15d14edc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:07.236970 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c3519-0bae-4735-83e7-9e15d14edc92 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:07.257384 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 860/1727] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:07.259830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c3519-0bae-4735-83e7-9e15d14edc92 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:07.262264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7d6c3519-0bae-4735-83e7-9e15d14edc92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:07.280702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-78992692-8669-4c11-b3a8-9edac935ccfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c9f163a3764a30b7f938b362900b69'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:07.297235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c3519-0bae-4735-83e7-9e15d14edc92 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:07.305262 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-78992692-8669-4c11-b3a8-9edac935ccfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c9f163a3764a30b7f938b362900b69'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:07.305690 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a7df9-d531-4ebc-b8a6-ab46181ed81b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:07.306607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a7df9-d531-4ebc-b8a6-ab46181ed81b service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:07.306997 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a7df9-d531-4ebc-b8a6-ab46181ed81b service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:07.307196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-78992692-8669-4c11-b3a8-9edac935ccfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c9f163a3764a30b7f938b362900b69'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:07.307525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a7df9-d531-4ebc-b8a6-ab46181ed81b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:07.310659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7b4a7df9-d531-4ebc-b8a6-ab46181ed81b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:07.328504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-78992692-8669-4c11-b3a8-9edac935ccfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c9f163a3764a30b7f938b362900b69'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:07.349914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-78992692-8669-4c11-b3a8-9edac935ccfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c9f163a3764a30b7f938b362900b69'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:07.373716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 868/1728] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:07 2022] DELETE /identity/v3/users/82c9f163a3764a30b7f938b362900b69 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:07.375435 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334a9e1-2d43-47a9-b2dd-9de387ba337b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:07.375735 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334a9e1-2d43-47a9-b2dd-9de387ba337b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:07.376019 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334a9e1-2d43-47a9-b2dd-9de387ba337b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:07.436910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5334a9e1-2d43-47a9-b2dd-9de387ba337b None None] MFA Rules not processed for user `f2c5536eddd346d692822a2f3befb48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:07.438542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5334a9e1-2d43-47a9-b2dd-9de387ba337b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:07.529578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a7df9-d531-4ebc-b8a6-ab46181ed81b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:07.537305 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a7df9-d531-4ebc-b8a6-ab46181ed81b service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:07.545961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:07.552421 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:07.566838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 861/1729] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:07 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:07.568934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a7df9-d531-4ebc-b8a6-ab46181ed81b service nova] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:07.571609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7b4a7df9-d531-4ebc-b8a6-ab46181ed81b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:07.600130 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 869/1730] 10.222.0.47 () {58 vars in 1058 bytes} [Wed Aug 3 10:44:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:07.600946 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334a9e1-2d43-47a9-b2dd-9de387ba337b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:07.603140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5334a9e1-2d43-47a9-b2dd-9de387ba337b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:07.636478 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334a9e1-2d43-47a9-b2dd-9de387ba337b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:07.638996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a7df9-d531-4ebc-b8a6-ab46181ed81b service nova] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:07.648834 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a8f91-1d88-4a87-a246-221376de906e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2c5536eddd346d692822a2f3befb48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:07.650994 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a8f91-1d88-4a87-a246-221376de906e service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:07.652971 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a8f91-1d88-4a87-a246-221376de906e service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:07.653414 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a8f91-1d88-4a87-a246-221376de906e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:07.672234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-663a8f91-1d88-4a87-a246-221376de906e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:07.704136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a8f91-1d88-4a87-a246-221376de906e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:07.714533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a8f91-1d88-4a87-a246-221376de906e service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:07.769682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 870/1731] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:07.775054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5e08c-f1c7-4339-993c-9c9b88842de0 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:07.782247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5e08c-f1c7-4339-993c-9c9b88842de0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:07.782908 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5e08c-f1c7-4339-993c-9c9b88842de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:07.850228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beeb28f-1076-449b-9700-de13141e42ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:07.851627 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beeb28f-1076-449b-9700-de13141e42ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:07.852177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beeb28f-1076-449b-9700-de13141e42ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:07.852657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beeb28f-1076-449b-9700-de13141e42ca admin admin] PATH_INFO: `/v3/projects/7903f56e8e544aa6a71f3e24b7ebb674/users/0ee9804c767b41ef9c4d00908174f4d2/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:07.889938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-fca5e08c-f1c7-4339-993c-9c9b88842de0 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:07.899950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fca5e08c-f1c7-4339-993c-9c9b88842de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:07.918043 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 871/1732] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:07.922865 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5e08c-f1c7-4339-993c-9c9b88842de0 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:07.930612 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beeb28f-1076-449b-9700-de13141e42ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=7903f56e8e544aa6a71f3e24b7ebb674, user_id=0ee9804c767b41ef9c4d00908174f4d2, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:07.947068 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beeb28f-1076-449b-9700-de13141e42ca admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:07.958882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5e08c-f1c7-4339-993c-9c9b88842de0 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:07.976588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3ac79-6d68-4f2a-9aeb-007f377fbef3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2c5536eddd346d692822a2f3befb48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:07.979006 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3ac79-6d68-4f2a-9aeb-007f377fbef3 service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:07.980921 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3ac79-6d68-4f2a-9aeb-007f377fbef3 service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:07.982706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3ac79-6d68-4f2a-9aeb-007f377fbef3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:08.003029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3ac79-6d68-4f2a-9aeb-007f377fbef3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:08.006785 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3ac79-6d68-4f2a-9aeb-007f377fbef3 service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:08.022825 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 862/1733] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:07 2022] PUT /identity/v3/projects/7903f56e8e544aa6a71f3e24b7ebb674/users/0ee9804c767b41ef9c4d00908174f4d2/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:08.037100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 872/1734] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:08.038549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3ac79-6d68-4f2a-9aeb-007f377fbef3 service placement] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:08.041313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beeb28f-1076-449b-9700-de13141e42ca admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:08.051868 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8beeb28f-1076-449b-9700-de13141e42ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:08.079011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3ac79-6d68-4f2a-9aeb-007f377fbef3 service placement] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:08.414590 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34225812-4fab-43b4-99f3-fdd27964872f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:08.421417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34225812-4fab-43b4-99f3-fdd27964872f admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:08.429318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34225812-4fab-43b4-99f3-fdd27964872f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:08.432536 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34225812-4fab-43b4-99f3-fdd27964872f admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:08.436047 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34225812-4fab-43b4-99f3-fdd27964872f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:08.442875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34225812-4fab-43b4-99f3-fdd27964872f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:08.445069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beeb28f-1076-449b-9700-de13141e42ca admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:08.453906 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac7b1a-61ae-41a1-96d1-b8813a7e05fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:08.455833 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac7b1a-61ae-41a1-96d1-b8813a7e05fd service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:08.456121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac7b1a-61ae-41a1-96d1-b8813a7e05fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:08.456409 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac7b1a-61ae-41a1-96d1-b8813a7e05fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:08.490701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac7b1a-61ae-41a1-96d1-b8813a7e05fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:08.492258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac7b1a-61ae-41a1-96d1-b8813a7e05fd service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:08.521586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 873/1735] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 481 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:08.522704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34225812-4fab-43b4-99f3-fdd27964872f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:08.536127 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-34225812-4fab-43b4-99f3-fdd27964872f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:08.564684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 863/1736] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:08.570236 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac7b1a-61ae-41a1-96d1-b8813a7e05fd service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:08.663353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac7b1a-61ae-41a1-96d1-b8813a7e05fd service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:08.693654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4e619-2039-4ffd-85d8-14ceeccf9d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:08.694756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4e619-2039-4ffd-85d8-14ceeccf9d40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:08.695225 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4e619-2039-4ffd-85d8-14ceeccf9d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:08.701893 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4e619-2039-4ffd-85d8-14ceeccf9d40 admin admin] PATH_INFO: `/v3/projects/7903f56e8e544aa6a71f3e24b7ebb674/users/0ee9804c767b41ef9c4d00908174f4d2/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:08.715502 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4e619-2039-4ffd-85d8-14ceeccf9d40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7903f56e8e544aa6a71f3e24b7ebb674, user_id=0ee9804c767b41ef9c4d00908174f4d2, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:08.761052 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4e619-2039-4ffd-85d8-14ceeccf9d40 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:08.844615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 864/1737] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:08 2022] PUT /identity/v3/projects/7903f56e8e544aa6a71f3e24b7ebb674/users/0ee9804c767b41ef9c4d00908174f4d2/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:08.871084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67124223-35e2-4c07-bf6f-e8dc2bad05b1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:08.871504 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67124223-35e2-4c07-bf6f-e8dc2bad05b1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:08.872082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67124223-35e2-4c07-bf6f-e8dc2bad05b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:09.036555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-67124223-35e2-4c07-bf6f-e8dc2bad05b1 None None] MFA Rules not processed for user `0ee9804c767b41ef9c4d00908174f4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:09.038286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-67124223-35e2-4c07-bf6f-e8dc2bad05b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:09.050028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34225812-4fab-43b4-99f3-fdd27964872f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:09.229182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:09.237736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:09.285532 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 865/1738] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:09.295473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67124223-35e2-4c07-bf6f-e8dc2bad05b1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:09.298931 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-67124223-35e2-4c07-bf6f-e8dc2bad05b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:09.343394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90e88b-736f-4315-92ea-2882330f44d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:09.345311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90e88b-736f-4315-92ea-2882330f44d7 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:09.367865 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90e88b-736f-4315-92ea-2882330f44d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:09.369810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90e88b-736f-4315-92ea-2882330f44d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:09.387460 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-cc90e88b-736f-4315-92ea-2882330f44d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:09.571853 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67124223-35e2-4c07-bf6f-e8dc2bad05b1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:09.590800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62f12f-5ff2-4e2c-90ff-58d280bcc198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:09.591693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62f12f-5ff2-4e2c-90ff-58d280bcc198 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:09.592077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62f12f-5ff2-4e2c-90ff-58d280bcc198 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:09.592464 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62f12f-5ff2-4e2c-90ff-58d280bcc198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:09.640736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62f12f-5ff2-4e2c-90ff-58d280bcc198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:09.646191 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62f12f-5ff2-4e2c-90ff-58d280bcc198 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:09.690435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 866/1739] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:09.747166 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90e88b-736f-4315-92ea-2882330f44d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:09.749102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90e88b-736f-4315-92ea-2882330f44d7 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:09.784838 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 874/1740] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:08 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 1263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:10.526847 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62f12f-5ff2-4e2c-90ff-58d280bcc198 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:10.529276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fc62f12f-5ff2-4e2c-90ff-58d280bcc198 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:10.757795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62f12f-5ff2-4e2c-90ff-58d280bcc198 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:10.780924 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce984fa8-a433-4007-9876-21e963920d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:10.783760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce984fa8-a433-4007-9876-21e963920d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:10.784101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce984fa8-a433-4007-9876-21e963920d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:10.784388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce984fa8-a433-4007-9876-21e963920d7c admin admin] PATH_INFO: `/v3/projects/064ea3050ab546439218d9a1e8742e19` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:10.792882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce984fa8-a433-4007-9876-21e963920d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=064ea3050ab546439218d9a1e8742e19)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:10.795069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce984fa8-a433-4007-9876-21e963920d7c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:10.901495 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ce984fa8-a433-4007-9876-21e963920d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '064ea3050ab546439218d9a1e8742e19', 'request_id': 'req-ce984fa8-a433-4007-9876-21e963920d7c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:10.913987 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ce984fa8-a433-4007-9876-21e963920d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '064ea3050ab546439218d9a1e8742e19', 'request_id': 'req-ce984fa8-a433-4007-9876-21e963920d7c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:10.925594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ce984fa8-a433-4007-9876-21e963920d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '064ea3050ab546439218d9a1e8742e19', 'request_id': 'req-ce984fa8-a433-4007-9876-21e963920d7c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:10.946871 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ce984fa8-a433-4007-9876-21e963920d7c admin admin] The token cache is being invalidate because project 064ea3050ab546439218d9a1e8742e19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:10.947108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ce984fa8-a433-4007-9876-21e963920d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:10.948887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ce984fa8-a433-4007-9876-21e963920d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:10.951409 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 867/1741] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:10 2022] DELETE /identity/v3/projects/064ea3050ab546439218d9a1e8742e19 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:10.959469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90e88b-736f-4315-92ea-2882330f44d7 service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:10.978239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90e88b-736f-4315-92ea-2882330f44d7 service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:11.018799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce984fa8-a433-4007-9876-21e963920d7c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:11.020477 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ce984fa8-a433-4007-9876-21e963920d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:11.069264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:11.126233 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c2954-fa64-405f-aae0-198950a90645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:11.127332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c2954-fa64-405f-aae0-198950a90645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:11.130110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c2954-fa64-405f-aae0-198950a90645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:11.131274 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c2954-fa64-405f-aae0-198950a90645 admin admin] PATH_INFO: `/v3/users/fdc4f69a8e034f3eb3cce74c0ab4cdd8` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:11.143527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce984fa8-a433-4007-9876-21e963920d7c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:11.144621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c2954-fa64-405f-aae0-198950a90645 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdc4f69a8e034f3eb3cce74c0ab4cdd8)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:11.149818 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac1317-a4f5-4809-95de-389ba4496816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:11.150534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac1317-a4f5-4809-95de-389ba4496816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:11.150827 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac1317-a4f5-4809-95de-389ba4496816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:11.151117 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac1317-a4f5-4809-95de-389ba4496816 admin admin] PATH_INFO: `/v3/users/7c261a283dc74e2f8b10a4d1780294b2` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:11.152660 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac1317-a4f5-4809-95de-389ba4496816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c261a283dc74e2f8b10a4d1780294b2)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:11.155071 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c2954-fa64-405f-aae0-198950a90645 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:11.155263 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac1317-a4f5-4809-95de-389ba4496816 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:11.269268 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d1ac1317-a4f5-4809-95de-389ba4496816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c261a283dc74e2f8b10a4d1780294b2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:11.292181 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-836c2954-fa64-405f-aae0-198950a90645 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdc4f69a8e034f3eb3cce74c0ab4cdd8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:11.295258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d1ac1317-a4f5-4809-95de-389ba4496816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c261a283dc74e2f8b10a4d1780294b2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:11.318161 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d1ac1317-a4f5-4809-95de-389ba4496816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c261a283dc74e2f8b10a4d1780294b2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:11.319217 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d1ac1317-a4f5-4809-95de-389ba4496816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c261a283dc74e2f8b10a4d1780294b2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:11.321299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d1ac1317-a4f5-4809-95de-389ba4496816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c261a283dc74e2f8b10a4d1780294b2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:11.364345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-836c2954-fa64-405f-aae0-198950a90645 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdc4f69a8e034f3eb3cce74c0ab4cdd8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:11.366682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-836c2954-fa64-405f-aae0-198950a90645 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdc4f69a8e034f3eb3cce74c0ab4cdd8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:11.380411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 868/1742] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:11 2022] DELETE /identity/v3/users/7c261a283dc74e2f8b10a4d1780294b2 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:11.384956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b9899-c769-4ccb-babe-08dde8cb9c6f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:11.385523 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b9899-c769-4ccb-babe-08dde8cb9c6f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:11.386056 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b9899-c769-4ccb-babe-08dde8cb9c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:11.402875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-836c2954-fa64-405f-aae0-198950a90645 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdc4f69a8e034f3eb3cce74c0ab4cdd8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:11.433673 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-385b9899-c769-4ccb-babe-08dde8cb9c6f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:11.436593 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-836c2954-fa64-405f-aae0-198950a90645 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdc4f69a8e034f3eb3cce74c0ab4cdd8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:11.439171 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-385b9899-c769-4ccb-babe-08dde8cb9c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:11.473304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 875/1743] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:10 2022] DELETE /identity/v3/users/fdc4f69a8e034f3eb3cce74c0ab4cdd8 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:44:11.661095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 869/1744] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:11.715630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c2954-fa64-405f-aae0-198950a90645 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:11.721521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-836c2954-fa64-405f-aae0-198950a90645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:11.771266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:12.111712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c2954-fa64-405f-aae0-198950a90645 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:12.126940 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73abd8-816d-43df-b1ee-b2f33964cfb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:12.128059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73abd8-816d-43df-b1ee-b2f33964cfb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:12.128573 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73abd8-816d-43df-b1ee-b2f33964cfb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:12.129098 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73abd8-816d-43df-b1ee-b2f33964cfb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:12.502666 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73abd8-816d-43df-b1ee-b2f33964cfb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:12.510027 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73abd8-816d-43df-b1ee-b2f33964cfb2 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:12.587294 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f54e10-a3d4-4b27-af8e-a85466fdd96d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:12.589002 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f54e10-a3d4-4b27-af8e-a85466fdd96d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:12.589589 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f54e10-a3d4-4b27-af8e-a85466fdd96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:12.654561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 876/1745] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 944 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:12.790077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-07f54e10-a3d4-4b27-af8e-a85466fdd96d None None] MFA Rules not processed for user `a5e714aad25f4fda8acf767ad7e2490b`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:12.798949 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-07f54e10-a3d4-4b27-af8e-a85466fdd96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:13.084711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:13.203047 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 870/1746] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 618 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:13.809611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebb7a9-7615-449d-a756-5c1d27f36343 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:13.810790 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebb7a9-7615-449d-a756-5c1d27f36343 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:13.811231 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebb7a9-7615-449d-a756-5c1d27f36343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:13.980172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7aebb7a9-7615-449d-a756-5c1d27f36343 None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:13.982810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7aebb7a9-7615-449d-a756-5c1d27f36343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:14.125649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 877/1747] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 3 10:44:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:14.444280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f54e10-a3d4-4b27-af8e-a85466fdd96d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:14.446133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-07f54e10-a3d4-4b27-af8e-a85466fdd96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:14.476911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebb7a9-7615-449d-a756-5c1d27f36343 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:14.478249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f54e10-a3d4-4b27-af8e-a85466fdd96d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:14.478850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7aebb7a9-7615-449d-a756-5c1d27f36343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:14.493986 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249db37-0ae8-4753-9aa0-4b3ee46b30b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:14.496674 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249db37-0ae8-4753-9aa0-4b3ee46b30b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:14.496998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249db37-0ae8-4753-9aa0-4b3ee46b30b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:14.497286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249db37-0ae8-4753-9aa0-4b3ee46b30b7 admin admin] PATH_INFO: `/v3/projects/3b0e004676b447ac869f131e08fa8715` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:14.502831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1249db37-0ae8-4753-9aa0-4b3ee46b30b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b0e004676b447ac869f131e08fa8715)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:14.506387 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1249db37-0ae8-4753-9aa0-4b3ee46b30b7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:14.615990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:14.634876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1249db37-0ae8-4753-9aa0-4b3ee46b30b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b0e004676b447ac869f131e08fa8715', 'request_id': 'req-1249db37-0ae8-4753-9aa0-4b3ee46b30b7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:14.656379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1249db37-0ae8-4753-9aa0-4b3ee46b30b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b0e004676b447ac869f131e08fa8715', 'request_id': 'req-1249db37-0ae8-4753-9aa0-4b3ee46b30b7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:14.696818 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1249db37-0ae8-4753-9aa0-4b3ee46b30b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b0e004676b447ac869f131e08fa8715', 'request_id': 'req-1249db37-0ae8-4753-9aa0-4b3ee46b30b7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:14.747619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1249db37-0ae8-4753-9aa0-4b3ee46b30b7 admin admin] The token cache is being invalidate because project 3b0e004676b447ac869f131e08fa8715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:14.747934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1249db37-0ae8-4753-9aa0-4b3ee46b30b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:14.749300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1249db37-0ae8-4753-9aa0-4b3ee46b30b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:14.782013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 871/1748] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:14 2022] DELETE /identity/v3/projects/3b0e004676b447ac869f131e08fa8715 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:14.784482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5853660-7c5f-4292-94b2-a63eb9c92b34 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:14.793047 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5853660-7c5f-4292-94b2-a63eb9c92b34 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:14.796637 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5853660-7c5f-4292-94b2-a63eb9c92b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:14.867402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebb7a9-7615-449d-a756-5c1d27f36343 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:14.871875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a5853660-7c5f-4292-94b2-a63eb9c92b34 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:14.878238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a5853660-7c5f-4292-94b2-a63eb9c92b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:15.107438 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 872/1749] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:15.121580 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207f466-651d-44de-a588-ed04b4687b20 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:15.121867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207f466-651d-44de-a588-ed04b4687b20 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:15.121999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207f466-651d-44de-a588-ed04b4687b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:15.195668 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e207f466-651d-44de-a588-ed04b4687b20 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:15.196886 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e456df-32ca-4f7e-9649-462b84fd8c5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:15.200667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e456df-32ca-4f7e-9649-462b84fd8c5e service swift] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:15.201327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e456df-32ca-4f7e-9649-462b84fd8c5e service swift] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:15.201646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e456df-32ca-4f7e-9649-462b84fd8c5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:15.203286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e207f466-651d-44de-a588-ed04b4687b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:15.208144 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a6e456df-32ca-4f7e-9649-462b84fd8c5e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:15.228159 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 873/1750] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:15.231003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207f466-651d-44de-a588-ed04b4687b20 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:15.265927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207f466-651d-44de-a588-ed04b4687b20 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:15.280035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee335e-cb01-4ee6-a643-5ff74caf476f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:15.284091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee335e-cb01-4ee6-a643-5ff74caf476f admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:15.284565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee335e-cb01-4ee6-a643-5ff74caf476f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:15.288567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee335e-cb01-4ee6-a643-5ff74caf476f admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:15.289449 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee335e-cb01-4ee6-a643-5ff74caf476f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:44:15.289858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee335e-cb01-4ee6-a643-5ff74caf476f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:15.296316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee335e-cb01-4ee6-a643-5ff74caf476f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:15.357799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 874/1751] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:44:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:44:15.363246 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee335e-cb01-4ee6-a643-5ff74caf476f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:15.393133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee335e-cb01-4ee6-a643-5ff74caf476f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:15.407999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5313f5c-f217-4188-a390-78bfe037843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:15.409049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5313f5c-f217-4188-a390-78bfe037843e admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:15.411507 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5313f5c-f217-4188-a390-78bfe037843e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:15.412881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5313f5c-f217-4188-a390-78bfe037843e admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:15.430046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5313f5c-f217-4188-a390-78bfe037843e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:15.432180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5313f5c-f217-4188-a390-78bfe037843e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:15.459796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e456df-32ca-4f7e-9649-462b84fd8c5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:15.463465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e456df-32ca-4f7e-9649-462b84fd8c5e service swift] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:15.520254 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 875/1752] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:15 2022] POST /identity/v3/projects => generated 355 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:15.527090 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e5b1d-727d-4a8e-824e-7b743cb3c479 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:15.527920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e5b1d-727d-4a8e-824e-7b743cb3c479 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:15.528635 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e5b1d-727d-4a8e-824e-7b743cb3c479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:15.536650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 878/1753] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:44:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1060 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 10:44:15.563125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e456df-32ca-4f7e-9649-462b84fd8c5e service swift] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:15.590598 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e69e5b1d-727d-4a8e-824e-7b743cb3c479 None None] MFA Rules not processed for user `5744cccc5a224263b71c052dc783e103`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:15.593570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e69e5b1d-727d-4a8e-824e-7b743cb3c479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:15.704943 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e456df-32ca-4f7e-9649-462b84fd8c5e service swift] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:15.711000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:15.899562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 876/1754] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:15.955277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:16.136140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4596113-3e48-4d02-b792-05fd33b12056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:16.139054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4596113-3e48-4d02-b792-05fd33b12056 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:16.139462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4596113-3e48-4d02-b792-05fd33b12056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:16.140765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4596113-3e48-4d02-b792-05fd33b12056 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:16.145745 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4596113-3e48-4d02-b792-05fd33b12056 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:16.156813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4596113-3e48-4d02-b792-05fd33b12056 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:16.348084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e5b1d-727d-4a8e-824e-7b743cb3c479 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:16.359954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 879/1755] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:15 2022] POST /identity/v3/users => generated 278 bytes in 810 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:16.369196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e5b1d-727d-4a8e-824e-7b743cb3c479 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:16.392536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311e8d1-85c2-4f8d-8b09-a6fc31ed79e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:16.396183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311e8d1-85c2-4f8d-8b09-a6fc31ed79e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:16.397511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311e8d1-85c2-4f8d-8b09-a6fc31ed79e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:16.398332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311e8d1-85c2-4f8d-8b09-a6fc31ed79e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:16.401243 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311e8d1-85c2-4f8d-8b09-a6fc31ed79e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:16.406047 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311e8d1-85c2-4f8d-8b09-a6fc31ed79e2 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:16.507907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 877/1756] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:16.510630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4596113-3e48-4d02-b792-05fd33b12056 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:16.535440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4596113-3e48-4d02-b792-05fd33b12056 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:16.559261 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eddeca0-9a3d-4009-badc-89e5979b91de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:16.559938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddeca0-9a3d-4009-badc-89e5979b91de admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:16.560271 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddeca0-9a3d-4009-badc-89e5979b91de admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:16.560565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddeca0-9a3d-4009-badc-89e5979b91de admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:16.562080 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddeca0-9a3d-4009-badc-89e5979b91de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:16.565650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddeca0-9a3d-4009-badc-89e5979b91de admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:16.593041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 880/1757] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:44:16.598242 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311e8d1-85c2-4f8d-8b09-a6fc31ed79e2 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:16.615407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311e8d1-85c2-4f8d-8b09-a6fc31ed79e2 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:16.655682 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc40db4-fa8f-4e00-9f20-0f013efeab69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:16.657304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc40db4-fa8f-4e00-9f20-0f013efeab69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:16.659409 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc40db4-fa8f-4e00-9f20-0f013efeab69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:16.660018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc40db4-fa8f-4e00-9f20-0f013efeab69 admin admin] PATH_INFO: `/v3/projects/00f8ce38df2a42b9885dd55e71dc9581/users/044d53fda6a54840a1d0563a96118622/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:16.696057 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc40db4-fa8f-4e00-9f20-0f013efeab69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f8ce38df2a42b9885dd55e71dc9581, user_id=044d53fda6a54840a1d0563a96118622, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:16.706422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc40db4-fa8f-4e00-9f20-0f013efeab69 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:16.743868 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 878/1758] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:16 2022] PUT /identity/v3/projects/00f8ce38df2a42b9885dd55e71dc9581/users/044d53fda6a54840a1d0563a96118622/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:16.756880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805f9ac-7d68-4f78-9993-a355df08edef None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:16.757196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805f9ac-7d68-4f78-9993-a355df08edef None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:16.757491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805f9ac-7d68-4f78-9993-a355df08edef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:16.830107 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6805f9ac-7d68-4f78-9993-a355df08edef None None] MFA Rules not processed for user `044d53fda6a54840a1d0563a96118622`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:16.831576 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6805f9ac-7d68-4f78-9993-a355df08edef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:17.194053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc40db4-fa8f-4e00-9f20-0f013efeab69 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:17.197353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6dc40db4-fa8f-4e00-9f20-0f013efeab69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:17.281771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 881/1759] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 525 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:17.283866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805f9ac-7d68-4f78-9993-a355df08edef None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:17.440048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc40db4-fa8f-4e00-9f20-0f013efeab69 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:17.458301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c24c3-7d82-4461-b61f-8be35ad0f9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:17.462826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c24c3-7d82-4461-b61f-8be35ad0f9f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:17.463467 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c24c3-7d82-4461-b61f-8be35ad0f9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:17.463966 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c24c3-7d82-4461-b61f-8be35ad0f9f5 admin admin] PATH_INFO: `/v3/projects/d03ed41586994666b3b447d1ee316ef8` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:17.472130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c24c3-7d82-4461-b61f-8be35ad0f9f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d03ed41586994666b3b447d1ee316ef8)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:17.476934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c24c3-7d82-4461-b61f-8be35ad0f9f5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:17.599910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-085c24c3-7d82-4461-b61f-8be35ad0f9f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd03ed41586994666b3b447d1ee316ef8', 'request_id': 'req-085c24c3-7d82-4461-b61f-8be35ad0f9f5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:17.615391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-085c24c3-7d82-4461-b61f-8be35ad0f9f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd03ed41586994666b3b447d1ee316ef8', 'request_id': 'req-085c24c3-7d82-4461-b61f-8be35ad0f9f5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:17.634350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-085c24c3-7d82-4461-b61f-8be35ad0f9f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd03ed41586994666b3b447d1ee316ef8', 'request_id': 'req-085c24c3-7d82-4461-b61f-8be35ad0f9f5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:17.660382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-085c24c3-7d82-4461-b61f-8be35ad0f9f5 admin admin] The token cache is being invalidate because project d03ed41586994666b3b447d1ee316ef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:17.661764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-085c24c3-7d82-4461-b61f-8be35ad0f9f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:17.663387 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-085c24c3-7d82-4461-b61f-8be35ad0f9f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:17.668545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 879/1760] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:17 2022] DELETE /identity/v3/projects/d03ed41586994666b3b447d1ee316ef8 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:44:17.673523 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c24c3-7d82-4461-b61f-8be35ad0f9f5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:17.677073 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-085c24c3-7d82-4461-b61f-8be35ad0f9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:17.745397 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805f9ac-7d68-4f78-9993-a355df08edef None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:17.812531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:17.946834 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c24c3-7d82-4461-b61f-8be35ad0f9f5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:17.959200 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae7d4b-c7e9-44f0-be7d-7cc80e6067de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:17.959920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae7d4b-c7e9-44f0-be7d-7cc80e6067de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:17.960209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae7d4b-c7e9-44f0-be7d-7cc80e6067de admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:17.960777 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae7d4b-c7e9-44f0-be7d-7cc80e6067de admin admin] PATH_INFO: `/v3/users/0ee9804c767b41ef9c4d00908174f4d2` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:17.964833 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae7d4b-c7e9-44f0-be7d-7cc80e6067de admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ee9804c767b41ef9c4d00908174f4d2)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:17.968611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae7d4b-c7e9-44f0-be7d-7cc80e6067de admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:18.095972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8575f-baf2-46b3-b881-996e59bfaa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:18.096737 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8575f-baf2-46b3-b881-996e59bfaa9f admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:18.097022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8575f-baf2-46b3-b881-996e59bfaa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:18.103366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8575f-baf2-46b3-b881-996e59bfaa9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:18.106623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8575f-baf2-46b3-b881-996e59bfaa9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:18.114086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8575f-baf2-46b3-b881-996e59bfaa9f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:18.149767 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7dae7d4b-c7e9-44f0-be7d-7cc80e6067de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ee9804c767b41ef9c4d00908174f4d2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:18.174818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:18.175152 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7dae7d4b-c7e9-44f0-be7d-7cc80e6067de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ee9804c767b41ef9c4d00908174f4d2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:18.200441 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7dae7d4b-c7e9-44f0-be7d-7cc80e6067de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ee9804c767b41ef9c4d00908174f4d2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:18.201281 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7dae7d4b-c7e9-44f0-be7d-7cc80e6067de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ee9804c767b41ef9c4d00908174f4d2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:18.202747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7dae7d4b-c7e9-44f0-be7d-7cc80e6067de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ee9804c767b41ef9c4d00908174f4d2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:18.293037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 882/1761] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:17 2022] POST /identity/v3/projects => generated 357 bytes in 1010 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:18.294590 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8575f-baf2-46b3-b881-996e59bfaa9f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:18.304317 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-61b8575f-baf2-46b3-b881-996e59bfaa9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:18.324163 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 880/1762] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:17 2022] DELETE /identity/v3/users/0ee9804c767b41ef9c4d00908174f4d2 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:44:18.607076 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8575f-baf2-46b3-b881-996e59bfaa9f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:18.778199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cd475-b7eb-414e-9f2a-aa43aa806fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:18.779353 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cd475-b7eb-414e-9f2a-aa43aa806fef admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:18.779800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cd475-b7eb-414e-9f2a-aa43aa806fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:18.779993 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cd475-b7eb-414e-9f2a-aa43aa806fef admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:18.780742 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9cd475-b7eb-414e-9f2a-aa43aa806fef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:18.793606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9cd475-b7eb-414e-9f2a-aa43aa806fef admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:18.859479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 883/1763] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:18 2022] POST /identity/v3/users => generated 285 bytes in 566 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:18.865070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae7d4b-c7e9-44f0-be7d-7cc80e6067de admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:18.881329 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae7d4b-c7e9-44f0-be7d-7cc80e6067de admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:18.914830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5ebc7-0f76-419a-9613-0addc903d672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:18.915567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5ebc7-0f76-419a-9613-0addc903d672 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:18.915853 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5ebc7-0f76-419a-9613-0addc903d672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:18.916822 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5ebc7-0f76-419a-9613-0addc903d672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:18.918877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5ebc7-0f76-419a-9613-0addc903d672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:18.925942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5ebc7-0f76-419a-9613-0addc903d672 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:18.960287 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 881/1764] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:44:18.965273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cd475-b7eb-414e-9f2a-aa43aa806fef admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:18.982588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cd475-b7eb-414e-9f2a-aa43aa806fef admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:19.000357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61f9f2-a131-430b-91fd-35bb6029d0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:19.003748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61f9f2-a131-430b-91fd-35bb6029d0bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:19.009683 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61f9f2-a131-430b-91fd-35bb6029d0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:19.010234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61f9f2-a131-430b-91fd-35bb6029d0bc admin admin] PATH_INFO: `/v3/projects/98164a766974409198b55e0b754aff27/users/081b9ea718714c708df4f3662b7254e1/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:19.086212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61f9f2-a131-430b-91fd-35bb6029d0bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=98164a766974409198b55e0b754aff27, user_id=081b9ea718714c708df4f3662b7254e1, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:19.092184 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61f9f2-a131-430b-91fd-35bb6029d0bc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:19.147079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5ebc7-0f76-419a-9613-0addc903d672 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:19.148058 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 884/1765] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:18 2022] PUT /identity/v3/projects/98164a766974409198b55e0b754aff27/users/081b9ea718714c708df4f3662b7254e1/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:19.189996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5ebc7-0f76-419a-9613-0addc903d672 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:19.450784 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fe689-bb31-4fca-8197-528f6cbcbdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:19.463217 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fe689-bb31-4fca-8197-528f6cbcbdea admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:19.464474 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fe689-bb31-4fca-8197-528f6cbcbdea admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:19.465268 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fe689-bb31-4fca-8197-528f6cbcbdea admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:19.467292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fe689-bb31-4fca-8197-528f6cbcbdea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:19.473566 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fe689-bb31-4fca-8197-528f6cbcbdea admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:19.522254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61f9f2-a131-430b-91fd-35bb6029d0bc admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:19.546097 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 882/1766] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:19.570617 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61f9f2-a131-430b-91fd-35bb6029d0bc admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:19.597115 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2071fc-29b9-441a-8457-a53a0e2ea3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:19.599747 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2071fc-29b9-441a-8457-a53a0e2ea3a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:19.600368 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2071fc-29b9-441a-8457-a53a0e2ea3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:19.601028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2071fc-29b9-441a-8457-a53a0e2ea3a3 admin admin] PATH_INFO: `/v3/projects/98164a766974409198b55e0b754aff27/users/081b9ea718714c708df4f3662b7254e1/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:19.623871 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2071fc-29b9-441a-8457-a53a0e2ea3a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98164a766974409198b55e0b754aff27, user_id=081b9ea718714c708df4f3662b7254e1, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:19.629209 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2071fc-29b9-441a-8457-a53a0e2ea3a3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:19.681607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 885/1767] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:19 2022] PUT /identity/v3/projects/98164a766974409198b55e0b754aff27/users/081b9ea718714c708df4f3662b7254e1/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:19.710873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a0bce-0ef0-42bf-84d1-f76ece57760b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:19.710873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a0bce-0ef0-42bf-84d1-f76ece57760b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:19.710873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a0bce-0ef0-42bf-84d1-f76ece57760b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:19.849717 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f30a0bce-0ef0-42bf-84d1-f76ece57760b None None] MFA Rules not processed for user `081b9ea718714c708df4f3662b7254e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:19.851829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f30a0bce-0ef0-42bf-84d1-f76ece57760b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:20.033153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:20.136521 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 883/1768] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:20.155771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2071fc-29b9-441a-8457-a53a0e2ea3a3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:20.157984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9c2071fc-29b9-441a-8457-a53a0e2ea3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:20.265148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:20.277495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2071fc-29b9-441a-8457-a53a0e2ea3a3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:20.289014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c12e6c-fff2-4c2f-9d58-6c1cf0a403f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:20.290873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c12e6c-fff2-4c2f-9d58-6c1cf0a403f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:20.292238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c12e6c-fff2-4c2f-9d58-6c1cf0a403f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:20.293778 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c12e6c-fff2-4c2f-9d58-6c1cf0a403f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:20.330814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c12e6c-fff2-4c2f-9d58-6c1cf0a403f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:20.337426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c12e6c-fff2-4c2f-9d58-6c1cf0a403f1 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:20.378081 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 886/1769] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:20 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:20.755983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a0bce-0ef0-42bf-84d1-f76ece57760b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:20.788245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a0bce-0ef0-42bf-84d1-f76ece57760b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:20.818989 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beca129-304e-4ef2-8bb3-1c8fd11bc2a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:20.819580 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beca129-304e-4ef2-8bb3-1c8fd11bc2a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:20.819868 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beca129-304e-4ef2-8bb3-1c8fd11bc2a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:20.827533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beca129-304e-4ef2-8bb3-1c8fd11bc2a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:20.836292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3beca129-304e-4ef2-8bb3-1c8fd11bc2a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:21.126675 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beca129-304e-4ef2-8bb3-1c8fd11bc2a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:21.153952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beca129-304e-4ef2-8bb3-1c8fd11bc2a4 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:21.309041 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 884/1770] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:20 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:44:22.292176 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c12e6c-fff2-4c2f-9d58-6c1cf0a403f1 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:22.295829 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-10c12e6c-fff2-4c2f-9d58-6c1cf0a403f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:22.451801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c12e6c-fff2-4c2f-9d58-6c1cf0a403f1 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:22.462253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd0587-9a7a-491f-afcb-8d7a6764fbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:22.463996 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd0587-9a7a-491f-afcb-8d7a6764fbab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:22.465039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd0587-9a7a-491f-afcb-8d7a6764fbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:22.465920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd0587-9a7a-491f-afcb-8d7a6764fbab admin admin] PATH_INFO: `/v3/projects/7903f56e8e544aa6a71f3e24b7ebb674` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:22.470915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd0587-9a7a-491f-afcb-8d7a6764fbab admin admin] RBAC: Authorizing `identity:delete_project(project_id=7903f56e8e544aa6a71f3e24b7ebb674)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:22.475432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd0587-9a7a-491f-afcb-8d7a6764fbab admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:22.632407 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-27dd0587-9a7a-491f-afcb-8d7a6764fbab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7903f56e8e544aa6a71f3e24b7ebb674', 'request_id': 'req-27dd0587-9a7a-491f-afcb-8d7a6764fbab'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:22.668730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-27dd0587-9a7a-491f-afcb-8d7a6764fbab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7903f56e8e544aa6a71f3e24b7ebb674', 'request_id': 'req-27dd0587-9a7a-491f-afcb-8d7a6764fbab'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:22.690731 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-27dd0587-9a7a-491f-afcb-8d7a6764fbab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7903f56e8e544aa6a71f3e24b7ebb674', 'request_id': 'req-27dd0587-9a7a-491f-afcb-8d7a6764fbab'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:22.741243 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-27dd0587-9a7a-491f-afcb-8d7a6764fbab admin admin] The token cache is being invalidate because project 7903f56e8e544aa6a71f3e24b7ebb674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:22.741591 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-27dd0587-9a7a-491f-afcb-8d7a6764fbab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:22.746613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-27dd0587-9a7a-491f-afcb-8d7a6764fbab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:22.748855 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 887/1771] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:22 2022] DELETE /identity/v3/projects/7903f56e8e544aa6a71f3e24b7ebb674 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:22.759048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400879a-6172-40c3-8d9e-5bee11188f5e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:22.759655 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400879a-6172-40c3-8d9e-5bee11188f5e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:22.760969 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400879a-6172-40c3-8d9e-5bee11188f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:22.805438 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f400879a-6172-40c3-8d9e-5bee11188f5e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:22.808252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f400879a-6172-40c3-8d9e-5bee11188f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:22.927393 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 885/1772] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:22.943215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8f217-aa07-4777-a9dc-12a641c99f10 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:22.943543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8f217-aa07-4777-a9dc-12a641c99f10 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:22.943850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8f217-aa07-4777-a9dc-12a641c99f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:22.973330 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ddc8f217-aa07-4777-a9dc-12a641c99f10 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:22.974798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ddc8f217-aa07-4777-a9dc-12a641c99f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:22.990343 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 888/1773] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:22.996513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400879a-6172-40c3-8d9e-5bee11188f5e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:23.017277 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400879a-6172-40c3-8d9e-5bee11188f5e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:23.029303 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63194f0-6b94-4b32-ad25-8726a50530e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:23.030066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63194f0-6b94-4b32-ad25-8726a50530e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:23.030353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63194f0-6b94-4b32-ad25-8726a50530e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:23.030640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63194f0-6b94-4b32-ad25-8726a50530e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:23.031392 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63194f0-6b94-4b32-ad25-8726a50530e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:44:23.031702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63194f0-6b94-4b32-ad25-8726a50530e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:23.033355 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63194f0-6b94-4b32-ad25-8726a50530e8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:23.048257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 886/1774] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:44:23.055556 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8f217-aa07-4777-a9dc-12a641c99f10 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:23.067789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8f217-aa07-4777-a9dc-12a641c99f10 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:23.074012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac7740-fa6c-4265-94a2-510b60b3fe85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:23.074839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac7740-fa6c-4265-94a2-510b60b3fe85 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:23.075180 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac7740-fa6c-4265-94a2-510b60b3fe85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:23.075533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac7740-fa6c-4265-94a2-510b60b3fe85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:23.076279 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac7740-fa6c-4265-94a2-510b60b3fe85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:23.078418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac7740-fa6c-4265-94a2-510b60b3fe85 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:23.137645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 889/1775] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:23 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:44:23.143964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63194f0-6b94-4b32-ad25-8726a50530e8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:23.168889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63194f0-6b94-4b32-ad25-8726a50530e8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:23.277707 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:23.280771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0aa3a1-a72f-4f2e-a63f-3417962e4e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:23.281695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0aa3a1-a72f-4f2e-a63f-3417962e4e57 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:23.282205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0aa3a1-a72f-4f2e-a63f-3417962e4e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:23.282538 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0aa3a1-a72f-4f2e-a63f-3417962e4e57 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:23.283325 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0aa3a1-a72f-4f2e-a63f-3417962e4e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:23.286018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0aa3a1-a72f-4f2e-a63f-3417962e4e57 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:23.315480 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 887/1776] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:23 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:44:23.317197 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac7740-fa6c-4265-94a2-510b60b3fe85 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:23.328865 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac7740-fa6c-4265-94a2-510b60b3fe85 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:23.334122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de7008-e18b-403f-91e0-d36c3eb15128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:23.334687 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de7008-e18b-403f-91e0-d36c3eb15128 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:23.334970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de7008-e18b-403f-91e0-d36c3eb15128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:23.335245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de7008-e18b-403f-91e0-d36c3eb15128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:23.335965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de7008-e18b-403f-91e0-d36c3eb15128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:23.337677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de7008-e18b-403f-91e0-d36c3eb15128 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:23.349528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 890/1777] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:23.354243 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0aa3a1-a72f-4f2e-a63f-3417962e4e57 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:23.365632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0aa3a1-a72f-4f2e-a63f-3417962e4e57 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:23.370401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb504b6-b2eb-43a9-b9d6-2bbe06b7c859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:23.371121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb504b6-b2eb-43a9-b9d6-2bbe06b7c859 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:23.371407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb504b6-b2eb-43a9-b9d6-2bbe06b7c859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:23.371683 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb504b6-b2eb-43a9-b9d6-2bbe06b7c859 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:23.372389 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb504b6-b2eb-43a9-b9d6-2bbe06b7c859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:23.374111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb504b6-b2eb-43a9-b9d6-2bbe06b7c859 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:23.387149 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 888/1778] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:44:23.391759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de7008-e18b-403f-91e0-d36c3eb15128 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:23.403972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de7008-e18b-403f-91e0-d36c3eb15128 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:23.409005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ade7f2-0df9-4985-9177-57f444bd2b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:23.409739 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ade7f2-0df9-4985-9177-57f444bd2b23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:23.410518 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ade7f2-0df9-4985-9177-57f444bd2b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:23.410867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ade7f2-0df9-4985-9177-57f444bd2b23 admin admin] PATH_INFO: `/v3/projects/4a95dcf20958490aa6ffaf48af355f1f/users/f929ee4769cb46c5908c9b125be801c2/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:23.428494 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ade7f2-0df9-4985-9177-57f444bd2b23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a95dcf20958490aa6ffaf48af355f1f, user_id=f929ee4769cb46c5908c9b125be801c2, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:23.431304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ade7f2-0df9-4985-9177-57f444bd2b23 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:23.451537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 891/1779] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:23 2022] PUT /identity/v3/projects/4a95dcf20958490aa6ffaf48af355f1f/users/f929ee4769cb46c5908c9b125be801c2/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:23.464300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a0815-c3b4-4895-9b83-fe318e8e2d3e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:23.464585 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a0815-c3b4-4895-9b83-fe318e8e2d3e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:23.464861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a0815-c3b4-4895-9b83-fe318e8e2d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:23.500396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-100a0815-c3b4-4895-9b83-fe318e8e2d3e None None] MFA Rules not processed for user `f929ee4769cb46c5908c9b125be801c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:23.501702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-100a0815-c3b4-4895-9b83-fe318e8e2d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:23.583121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 889/1780] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:23.589374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ade7f2-0df9-4985-9177-57f444bd2b23 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:23.617050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:23.690372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ade7f2-0df9-4985-9177-57f444bd2b23 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:23.696821 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d422f6b-b7fd-4a48-964e-1432c69caef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:23.697866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d422f6b-b7fd-4a48-964e-1432c69caef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:23.698312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d422f6b-b7fd-4a48-964e-1432c69caef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:23.698763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d422f6b-b7fd-4a48-964e-1432c69caef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:23.699901 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d422f6b-b7fd-4a48-964e-1432c69caef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:23.703433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d422f6b-b7fd-4a48-964e-1432c69caef1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:23.740727 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 892/1781] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:23 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:23.747256 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a0815-c3b4-4895-9b83-fe318e8e2d3e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:23.760772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a0815-c3b4-4895-9b83-fe318e8e2d3e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:23.852712 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f663650-8096-4c39-945a-87af95724dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:23.853985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f663650-8096-4c39-945a-87af95724dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:23.854491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f663650-8096-4c39-945a-87af95724dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:23.854991 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f663650-8096-4c39-945a-87af95724dc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:23.856239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f663650-8096-4c39-945a-87af95724dc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:23.860416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f663650-8096-4c39-945a-87af95724dc3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:23.908495 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 890/1782] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:23 2022] POST /identity/v3/users => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:23.911033 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d422f6b-b7fd-4a48-964e-1432c69caef1 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:23.932169 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d422f6b-b7fd-4a48-964e-1432c69caef1 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:23.940921 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037277b-1b27-459f-ae2b-f2c21b94f262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:23.942233 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037277b-1b27-459f-ae2b-f2c21b94f262 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:23.942867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037277b-1b27-459f-ae2b-f2c21b94f262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:23.943737 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037277b-1b27-459f-ae2b-f2c21b94f262 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:23.945307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037277b-1b27-459f-ae2b-f2c21b94f262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:23.948750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037277b-1b27-459f-ae2b-f2c21b94f262 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:23.973085 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 893/1783] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:23.975353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f663650-8096-4c39-945a-87af95724dc3 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:23.993759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f663650-8096-4c39-945a-87af95724dc3 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:24.001574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21f201-f792-43b9-8606-0b562ced0b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:24.003356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21f201-f792-43b9-8606-0b562ced0b40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:24.005258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21f201-f792-43b9-8606-0b562ced0b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:24.005770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21f201-f792-43b9-8606-0b562ced0b40 admin admin] PATH_INFO: `/v3/projects/bdf2b915055d47bdbc5f035fb161ac15/users/8f6bbd0eaa654f349f3f1e5cdc62c35b/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:24.035471 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21f201-f792-43b9-8606-0b562ced0b40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf2b915055d47bdbc5f035fb161ac15, user_id=8f6bbd0eaa654f349f3f1e5cdc62c35b, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:24.047994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21f201-f792-43b9-8606-0b562ced0b40 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:24.071436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 891/1784] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:23 2022] PUT /identity/v3/projects/bdf2b915055d47bdbc5f035fb161ac15/users/8f6bbd0eaa654f349f3f1e5cdc62c35b/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:24.075916 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037277b-1b27-459f-ae2b-f2c21b94f262 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:24.088298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037277b-1b27-459f-ae2b-f2c21b94f262 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:24.236221 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c356e05-5a24-4884-a5a4-c934fd5442e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:24.237150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c356e05-5a24-4884-a5a4-c934fd5442e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:24.255848 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c356e05-5a24-4884-a5a4-c934fd5442e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:24.256542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c356e05-5a24-4884-a5a4-c934fd5442e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:24.257558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c356e05-5a24-4884-a5a4-c934fd5442e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:24.262645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c356e05-5a24-4884-a5a4-c934fd5442e0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:24.318129 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 894/1785] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:44:24.328566 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21f201-f792-43b9-8606-0b562ced0b40 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:24.377062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21f201-f792-43b9-8606-0b562ced0b40 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:24.418083 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c91b7e-7228-47e6-aa92-1ce6a9750ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:24.421888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c91b7e-7228-47e6-aa92-1ce6a9750ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:24.423354 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c91b7e-7228-47e6-aa92-1ce6a9750ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:24.423905 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c91b7e-7228-47e6-aa92-1ce6a9750ed7 admin admin] PATH_INFO: `/v3/projects/bdf2b915055d47bdbc5f035fb161ac15/users/8f6bbd0eaa654f349f3f1e5cdc62c35b/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:24.442774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c91b7e-7228-47e6-aa92-1ce6a9750ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf2b915055d47bdbc5f035fb161ac15, user_id=8f6bbd0eaa654f349f3f1e5cdc62c35b, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:24.454317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c91b7e-7228-47e6-aa92-1ce6a9750ed7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:24.502167 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 892/1786] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:24 2022] PUT /identity/v3/projects/bdf2b915055d47bdbc5f035fb161ac15/users/8f6bbd0eaa654f349f3f1e5cdc62c35b/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:24.530768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f72ce-56cf-4be2-9961-0f8020bffd00 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:24.531642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f72ce-56cf-4be2-9961-0f8020bffd00 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:24.532172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f72ce-56cf-4be2-9961-0f8020bffd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:24.644585 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4a2f72ce-56cf-4be2-9961-0f8020bffd00 None None] MFA Rules not processed for user `8f6bbd0eaa654f349f3f1e5cdc62c35b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:24.648036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4a2f72ce-56cf-4be2-9961-0f8020bffd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:24.850537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 895/1787] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:24.869802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c91b7e-7228-47e6-aa92-1ce6a9750ed7 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:24.871713 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-22c91b7e-7228-47e6-aa92-1ce6a9750ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:24.912475 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:25.012266 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c91b7e-7228-47e6-aa92-1ce6a9750ed7 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:25.024121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d73d4-c0c3-48c9-b276-805077d330c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:25.024856 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d73d4-c0c3-48c9-b276-805077d330c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:25.025232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d73d4-c0c3-48c9-b276-805077d330c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:25.025485 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d73d4-c0c3-48c9-b276-805077d330c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:25.160635 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d73d4-c0c3-48c9-b276-805077d330c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:25.162130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d73d4-c0c3-48c9-b276-805077d330c8 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:25.190289 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 893/1788] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:25.435652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a30a4-f8a0-495f-89e4-c6535f6290f3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:25.436614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a30a4-f8a0-495f-89e4-c6535f6290f3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:25.436614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a30a4-f8a0-495f-89e4-c6535f6290f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:25.489827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8f6a30a4-f8a0-495f-89e4-c6535f6290f3 None None] MFA Rules not processed for user `1c211cc19a2045bd9383d31c7e81ceac`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:25.491317 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8f6a30a4-f8a0-495f-89e4-c6535f6290f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:25.515011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:25.695377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 896/1789] 10.222.0.61 () {58 vars in 1015 bytes} [Wed Aug 3 10:44:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:25.805361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d73d4-c0c3-48c9-b276-805077d330c8 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:25.805684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a30a4-f8a0-495f-89e4-c6535f6290f3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:25.813084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5e6d73d4-c0c3-48c9-b276-805077d330c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:25.813431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8f6a30a4-f8a0-495f-89e4-c6535f6290f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:25.867351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d73d4-c0c3-48c9-b276-805077d330c8 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:25.885323 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a30a4-f8a0-495f-89e4-c6535f6290f3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:25.911808 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26832194-c1d8-4f0b-9f8c-f6242e80f9f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:25.912439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821f65d-2c22-4d56-9d1b-26e705b42390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:25.914819 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821f65d-2c22-4d56-9d1b-26e705b42390 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:25.915704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26832194-c1d8-4f0b-9f8c-f6242e80f9f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:25.915959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821f65d-2c22-4d56-9d1b-26e705b42390 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:25.916299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821f65d-2c22-4d56-9d1b-26e705b42390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:25.916591 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26832194-c1d8-4f0b-9f8c-f6242e80f9f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:25.918096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26832194-c1d8-4f0b-9f8c-f6242e80f9f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:25.958599 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26832194-c1d8-4f0b-9f8c-f6242e80f9f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:25.963209 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821f65d-2c22-4d56-9d1b-26e705b42390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:25.967195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26832194-c1d8-4f0b-9f8c-f6242e80f9f5 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:25.968244 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821f65d-2c22-4d56-9d1b-26e705b42390 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:26.002672 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 897/1790] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:26.006124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821f65d-2c22-4d56-9d1b-26e705b42390 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:26.033645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 894/1791] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:26.041262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26832194-c1d8-4f0b-9f8c-f6242e80f9f5 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:26.042081 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821f65d-2c22-4d56-9d1b-26e705b42390 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:26.053009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-26832194-c1d8-4f0b-9f8c-f6242e80f9f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:26.055268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da9436-e238-4267-90cf-3eee3fe260ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:26.057847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da9436-e238-4267-90cf-3eee3fe260ea service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:26.058359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da9436-e238-4267-90cf-3eee3fe260ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:26.059290 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da9436-e238-4267-90cf-3eee3fe260ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:26.090776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da9436-e238-4267-90cf-3eee3fe260ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:26.097187 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da9436-e238-4267-90cf-3eee3fe260ea service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:26.116372 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26832194-c1d8-4f0b-9f8c-f6242e80f9f5 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:26.133575 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 898/1792] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:26.139879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52dc41-6e5f-474a-8e27-f7e80c33dd4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:26.155068 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52dc41-6e5f-474a-8e27-f7e80c33dd4f service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:26.155559 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52dc41-6e5f-474a-8e27-f7e80c33dd4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:26.156315 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52dc41-6e5f-474a-8e27-f7e80c33dd4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:26.200939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52dc41-6e5f-474a-8e27-f7e80c33dd4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:26.204446 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52dc41-6e5f-474a-8e27-f7e80c33dd4f service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:26.264814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 895/1793] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:29.344288 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da9436-e238-4267-90cf-3eee3fe260ea service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:29.345063 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-71da9436-e238-4267-90cf-3eee3fe260ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:29.597451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da9436-e238-4267-90cf-3eee3fe260ea service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:29.616261 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f45f89-4db3-416b-b1f2-b228b43568a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:29.617263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f45f89-4db3-416b-b1f2-b228b43568a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:29.618716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f45f89-4db3-416b-b1f2-b228b43568a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:29.619329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f45f89-4db3-416b-b1f2-b228b43568a0 admin admin] PATH_INFO: `/v3/users/22c851fa8799404c9089852e3e6b48e7` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:29.623918 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f45f89-4db3-416b-b1f2-b228b43568a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22c851fa8799404c9089852e3e6b48e7)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:29.628014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f45f89-4db3-416b-b1f2-b228b43568a0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:29.780134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e9f45f89-4db3-416b-b1f2-b228b43568a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22c851fa8799404c9089852e3e6b48e7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:29.824976 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e9f45f89-4db3-416b-b1f2-b228b43568a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22c851fa8799404c9089852e3e6b48e7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:29.826753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e9f45f89-4db3-416b-b1f2-b228b43568a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22c851fa8799404c9089852e3e6b48e7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:29.845459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e9f45f89-4db3-416b-b1f2-b228b43568a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22c851fa8799404c9089852e3e6b48e7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:29.867147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e9f45f89-4db3-416b-b1f2-b228b43568a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22c851fa8799404c9089852e3e6b48e7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:29.886854 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 899/1794] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:29 2022] DELETE /identity/v3/users/22c851fa8799404c9089852e3e6b48e7 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:29.891488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d8559-c31c-46d8-a308-fe8b3a8fb423 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:29.893118 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d8559-c31c-46d8-a308-fe8b3a8fb423 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:29.893788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d8559-c31c-46d8-a308-fe8b3a8fb423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:29.938826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a61d8559-c31c-46d8-a308-fe8b3a8fb423 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:29.940925 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a61d8559-c31c-46d8-a308-fe8b3a8fb423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:30.146004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 896/1795] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:30.492083 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f45f89-4db3-416b-b1f2-b228b43568a0 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:30.494850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e9f45f89-4db3-416b-b1f2-b228b43568a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:30.619448 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:30.868279 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f45f89-4db3-416b-b1f2-b228b43568a0 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:30.881071 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49dfb6-15df-4afe-8c96-0e0b6855aed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:30.882051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49dfb6-15df-4afe-8c96-0e0b6855aed3 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:30.882580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49dfb6-15df-4afe-8c96-0e0b6855aed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:30.883018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49dfb6-15df-4afe-8c96-0e0b6855aed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:30.913253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49dfb6-15df-4afe-8c96-0e0b6855aed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:30.915011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49dfb6-15df-4afe-8c96-0e0b6855aed3 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:30.940795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 900/1796] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:31.851790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f462be-888e-49ea-a68d-ffd70aa42f2c None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:31.852756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f462be-888e-49ea-a68d-ffd70aa42f2c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:31.852756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f462be-888e-49ea-a68d-ffd70aa42f2c None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:31.855878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 897/1797] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 3 10:44:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 10:44:32.052101 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c65820-ebd6-4baf-933b-99c3c11d6224 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:32.053764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c65820-ebd6-4baf-933b-99c3c11d6224 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:32.054470 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c65820-ebd6-4baf-933b-99c3c11d6224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:32.095251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f462be-888e-49ea-a68d-ffd70aa42f2c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:32.098293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-77f462be-888e-49ea-a68d-ffd70aa42f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:32.143056 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-60c65820-ebd6-4baf-933b-99c3c11d6224 None None] MFA Rules not processed for user `e66f25cc329f418eba6bd637bf3e8a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:32.152469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-60c65820-ebd6-4baf-933b-99c3c11d6224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:32.270712 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:32.322474 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f462be-888e-49ea-a68d-ffd70aa42f2c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:32.333307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27283fd-c12a-4a08-88cb-bc1ac32b7a7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:32.336350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27283fd-c12a-4a08-88cb-bc1ac32b7a7e service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:32.336721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27283fd-c12a-4a08-88cb-bc1ac32b7a7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:32.337040 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27283fd-c12a-4a08-88cb-bc1ac32b7a7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:32.346406 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e27283fd-c12a-4a08-88cb-bc1ac32b7a7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:32.401974 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 901/1798] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 3 10:44:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:32.466106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27283fd-c12a-4a08-88cb-bc1ac32b7a7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:32.469135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27283fd-c12a-4a08-88cb-bc1ac32b7a7e service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:32.532890 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 898/1799] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:32 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:33.045677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3f1bb-3a5a-46f3-a994-04b8ffeb282c None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:33.046271 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3f1bb-3a5a-46f3-a994-04b8ffeb282c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:33.046717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3f1bb-3a5a-46f3-a994-04b8ffeb282c None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:33.054524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 902/1800] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 3 10:44:33 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:44:33.234224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0cc0d-4c82-43b4-a88b-652b743941fe None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:33.235111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0cc0d-4c82-43b4-a88b-652b743941fe None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:33.235111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0cc0d-4c82-43b4-a88b-652b743941fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:33.307543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-dbe0cc0d-4c82-43b4-a88b-652b743941fe None None] MFA Rules not processed for user `e66f25cc329f418eba6bd637bf3e8a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:33.309342 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-dbe0cc0d-4c82-43b4-a88b-652b743941fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:33.348956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 899/1801] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 3 10:44:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:34.110647 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3f1bb-3a5a-46f3-a994-04b8ffeb282c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:34.115628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a3c3f1bb-3a5a-46f3-a994-04b8ffeb282c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:34.181577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3f1bb-3a5a-46f3-a994-04b8ffeb282c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:34.200776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e45577-ea9c-46ad-9c1e-62208d54bd38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:34.202628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e45577-ea9c-46ad-9c1e-62208d54bd38 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:34.204165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e45577-ea9c-46ad-9c1e-62208d54bd38 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:34.207208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e45577-ea9c-46ad-9c1e-62208d54bd38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:34.236278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e45577-ea9c-46ad-9c1e-62208d54bd38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:34.241111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e45577-ea9c-46ad-9c1e-62208d54bd38 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:34.299007 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 903/1802] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:44:34.396754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0cc0d-4c82-43b4-a88b-652b743941fe None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:34.401909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-dbe0cc0d-4c82-43b4-a88b-652b743941fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:34.433558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0cc0d-4c82-43b4-a88b-652b743941fe None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:34.445488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70444513-b328-4eb0-95cd-1f50a1d463e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:34.446548 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70444513-b328-4eb0-95cd-1f50a1d463e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:34.448214 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70444513-b328-4eb0-95cd-1f50a1d463e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:34.448768 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70444513-b328-4eb0-95cd-1f50a1d463e5 admin admin] PATH_INFO: `/v3/projects/2f75605309684b169385b6ea4da9f14a` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:34.451360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70444513-b328-4eb0-95cd-1f50a1d463e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f75605309684b169385b6ea4da9f14a)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:34.455134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70444513-b328-4eb0-95cd-1f50a1d463e5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:34.489863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e45577-ea9c-46ad-9c1e-62208d54bd38 service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:34.516687 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e45577-ea9c-46ad-9c1e-62208d54bd38 service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:34.554533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70444513-b328-4eb0-95cd-1f50a1d463e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f75605309684b169385b6ea4da9f14a', 'request_id': 'req-70444513-b328-4eb0-95cd-1f50a1d463e5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:34.565836 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70444513-b328-4eb0-95cd-1f50a1d463e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f75605309684b169385b6ea4da9f14a', 'request_id': 'req-70444513-b328-4eb0-95cd-1f50a1d463e5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:34.580694 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70444513-b328-4eb0-95cd-1f50a1d463e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f75605309684b169385b6ea4da9f14a', 'request_id': 'req-70444513-b328-4eb0-95cd-1f50a1d463e5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:34.608737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70444513-b328-4eb0-95cd-1f50a1d463e5 admin admin] The token cache is being invalidate because project 2f75605309684b169385b6ea4da9f14a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:34.609421 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70444513-b328-4eb0-95cd-1f50a1d463e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:34.611710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70444513-b328-4eb0-95cd-1f50a1d463e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:34.619571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 900/1803] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:34 2022] DELETE /identity/v3/projects/2f75605309684b169385b6ea4da9f14a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:34.621702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70444513-b328-4eb0-95cd-1f50a1d463e5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:34.624503 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-70444513-b328-4eb0-95cd-1f50a1d463e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:34.796269 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78500206-0e3f-4949-a1b5-ae6f3a292d46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:34.797573 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78500206-0e3f-4949-a1b5-ae6f3a292d46 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:34.798473 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78500206-0e3f-4949-a1b5-ae6f3a292d46 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:34.799000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78500206-0e3f-4949-a1b5-ae6f3a292d46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:34.805209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70444513-b328-4eb0-95cd-1f50a1d463e5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:34.805507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-78500206-0e3f-4949-a1b5-ae6f3a292d46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:34.838564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab6d25-897c-4230-a3d4-d84823dbea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:34.839267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab6d25-897c-4230-a3d4-d84823dbea8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:34.839558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab6d25-897c-4230-a3d4-d84823dbea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:34.839835 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab6d25-897c-4230-a3d4-d84823dbea8f admin admin] PATH_INFO: `/v3/users/a08a8859e0ee4d1cad4c9ae770b049d0` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:34.841188 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab6d25-897c-4230-a3d4-d84823dbea8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a08a8859e0ee4d1cad4c9ae770b049d0)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:34.841681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78500206-0e3f-4949-a1b5-ae6f3a292d46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:34.843197 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab6d25-897c-4230-a3d4-d84823dbea8f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:34.847262 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78500206-0e3f-4949-a1b5-ae6f3a292d46 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:34.860367 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:34.907805 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 904/1804] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:34.959668 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a4ab6d25-897c-4230-a3d4-d84823dbea8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a08a8859e0ee4d1cad4c9ae770b049d0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:34.979811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a4ab6d25-897c-4230-a3d4-d84823dbea8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a08a8859e0ee4d1cad4c9ae770b049d0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:35.007866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a4ab6d25-897c-4230-a3d4-d84823dbea8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a08a8859e0ee4d1cad4c9ae770b049d0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:35.012285 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a4ab6d25-897c-4230-a3d4-d84823dbea8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a08a8859e0ee4d1cad4c9ae770b049d0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:35.013793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a4ab6d25-897c-4230-a3d4-d84823dbea8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a08a8859e0ee4d1cad4c9ae770b049d0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:35.062615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 901/1805] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:34 2022] DELETE /identity/v3/users/a08a8859e0ee4d1cad4c9ae770b049d0 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:36.381271 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78500206-0e3f-4949-a1b5-ae6f3a292d46 service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:36.400432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78500206-0e3f-4949-a1b5-ae6f3a292d46 service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:36.475001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:36.606550 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680f018-3fc6-402b-b0cf-a32b8b5b08c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:36.607691 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680f018-3fc6-402b-b0cf-a32b8b5b08c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:36.609030 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680f018-3fc6-402b-b0cf-a32b8b5b08c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:36.610100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680f018-3fc6-402b-b0cf-a32b8b5b08c0 admin admin] PATH_INFO: `/v3/projects/151af499b5cb4801ace50e3b4c732c10` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:36.612687 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680f018-3fc6-402b-b0cf-a32b8b5b08c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=151af499b5cb4801ace50e3b4c732c10)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:36.620247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680f018-3fc6-402b-b0cf-a32b8b5b08c0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:36.783200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9680f018-3fc6-402b-b0cf-a32b8b5b08c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '151af499b5cb4801ace50e3b4c732c10', 'request_id': 'req-9680f018-3fc6-402b-b0cf-a32b8b5b08c0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:36.798864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9680f018-3fc6-402b-b0cf-a32b8b5b08c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '151af499b5cb4801ace50e3b4c732c10', 'request_id': 'req-9680f018-3fc6-402b-b0cf-a32b8b5b08c0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:36.809624 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9680f018-3fc6-402b-b0cf-a32b8b5b08c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '151af499b5cb4801ace50e3b4c732c10', 'request_id': 'req-9680f018-3fc6-402b-b0cf-a32b8b5b08c0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:36.845611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9680f018-3fc6-402b-b0cf-a32b8b5b08c0 admin admin] The token cache is being invalidate because project 151af499b5cb4801ace50e3b4c732c10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:36.846311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9680f018-3fc6-402b-b0cf-a32b8b5b08c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:36.848005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9680f018-3fc6-402b-b0cf-a32b8b5b08c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:36.851056 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 905/1806] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:36 2022] DELETE /identity/v3/projects/151af499b5cb4801ace50e3b4c732c10 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:36.860717 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53912c9f-09d1-4162-93f7-cd9a79dcae2d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:36.862260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53912c9f-09d1-4162-93f7-cd9a79dcae2d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:36.864454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53912c9f-09d1-4162-93f7-cd9a79dcae2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:36.895403 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-53912c9f-09d1-4162-93f7-cd9a79dcae2d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:36.897371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-53912c9f-09d1-4162-93f7-cd9a79dcae2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:37.024774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 902/1807] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:37.027996 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d60e5-cc71-4334-9868-a17d15191284 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:37.028771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d60e5-cc71-4334-9868-a17d15191284 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:37.029428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d60e5-cc71-4334-9868-a17d15191284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:37.072937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-260d60e5-cc71-4334-9868-a17d15191284 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:37.076864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-260d60e5-cc71-4334-9868-a17d15191284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:37.109194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53912c9f-09d1-4162-93f7-cd9a79dcae2d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:37.109392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 906/1808] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:37.133060 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53912c9f-09d1-4162-93f7-cd9a79dcae2d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:37.139826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea85d0-2656-410c-9246-f2bc2735e51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:37.140576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea85d0-2656-410c-9246-f2bc2735e51a admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:37.140922 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea85d0-2656-410c-9246-f2bc2735e51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:37.141264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea85d0-2656-410c-9246-f2bc2735e51a admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:37.142157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea85d0-2656-410c-9246-f2bc2735e51a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:44:37.142669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea85d0-2656-410c-9246-f2bc2735e51a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:37.145308 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea85d0-2656-410c-9246-f2bc2735e51a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:37.163374 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 903/1809] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:44:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:44:37.170518 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d60e5-cc71-4334-9868-a17d15191284 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:37.196730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d60e5-cc71-4334-9868-a17d15191284 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:37.205238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab2a7e-9a4c-4c86-9529-9d93906abdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:37.208262 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab2a7e-9a4c-4c86-9529-9d93906abdae admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:37.209817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab2a7e-9a4c-4c86-9529-9d93906abdae admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:37.210569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab2a7e-9a4c-4c86-9529-9d93906abdae admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:37.212143 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab2a7e-9a4c-4c86-9529-9d93906abdae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:37.215516 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab2a7e-9a4c-4c86-9529-9d93906abdae admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:37.267604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 907/1810] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:37 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:37.273747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea85d0-2656-410c-9246-f2bc2735e51a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:37.288975 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea85d0-2656-410c-9246-f2bc2735e51a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:37.447957 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e649d30-ee1e-440f-ad00-faffb53e828b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:37.450070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e649d30-ee1e-440f-ad00-faffb53e828b admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:37.450612 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e649d30-ee1e-440f-ad00-faffb53e828b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:37.450973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e649d30-ee1e-440f-ad00-faffb53e828b admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:37.452587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e649d30-ee1e-440f-ad00-faffb53e828b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:37.454241 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e649d30-ee1e-440f-ad00-faffb53e828b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:37.493106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 904/1811] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:37 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:37.496438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab2a7e-9a4c-4c86-9529-9d93906abdae admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:37.509705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab2a7e-9a4c-4c86-9529-9d93906abdae admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:37.515761 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd4bcf-6b06-4778-b822-03ce5c13e55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:37.516517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd4bcf-6b06-4778-b822-03ce5c13e55b admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:37.516787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd4bcf-6b06-4778-b822-03ce5c13e55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:37.517190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd4bcf-6b06-4778-b822-03ce5c13e55b admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:37.518039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd4bcf-6b06-4778-b822-03ce5c13e55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:37.520493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd4bcf-6b06-4778-b822-03ce5c13e55b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:37.536872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 908/1812] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:37.541877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e649d30-ee1e-440f-ad00-faffb53e828b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:37.551113 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:37.562483 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e649d30-ee1e-440f-ad00-faffb53e828b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:37.573573 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa9a68-c1d6-498f-bec9-0976aacb8966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:37.574964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa9a68-c1d6-498f-bec9-0976aacb8966 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:37.575425 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa9a68-c1d6-498f-bec9-0976aacb8966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:37.575880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa9a68-c1d6-498f-bec9-0976aacb8966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:37.577111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa9a68-c1d6-498f-bec9-0976aacb8966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:37.579007 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa9a68-c1d6-498f-bec9-0976aacb8966 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:37.592805 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 905/1813] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:44:37.598032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd4bcf-6b06-4778-b822-03ce5c13e55b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:37.609717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd4bcf-6b06-4778-b822-03ce5c13e55b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:37.615622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023eb6b-b41b-4f6d-8029-88a976eaf8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:37.616437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023eb6b-b41b-4f6d-8029-88a976eaf8d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:37.616796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023eb6b-b41b-4f6d-8029-88a976eaf8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:37.617137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023eb6b-b41b-4f6d-8029-88a976eaf8d6 admin admin] PATH_INFO: `/v3/projects/4af1e50bdd9849f9b0066f134623237b/users/649d80caa93b48e9b9d989a31b5d476c/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:37.634832 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023eb6b-b41b-4f6d-8029-88a976eaf8d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af1e50bdd9849f9b0066f134623237b, user_id=649d80caa93b48e9b9d989a31b5d476c, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:37.637628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023eb6b-b41b-4f6d-8029-88a976eaf8d6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:37.657307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 909/1814] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:37 2022] PUT /identity/v3/projects/4af1e50bdd9849f9b0066f134623237b/users/649d80caa93b48e9b9d989a31b5d476c/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:37.669715 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad376a8-9ebc-4c6e-8e44-024651942797 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:37.669999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad376a8-9ebc-4c6e-8e44-024651942797 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:37.670263 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad376a8-9ebc-4c6e-8e44-024651942797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:37.707722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-dad376a8-9ebc-4c6e-8e44-024651942797 None None] MFA Rules not processed for user `649d80caa93b48e9b9d989a31b5d476c`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:37.709121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-dad376a8-9ebc-4c6e-8e44-024651942797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:37.804703 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 906/1815] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:37.810110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023eb6b-b41b-4f6d-8029-88a976eaf8d6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:37.872615 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:37.889680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023eb6b-b41b-4f6d-8029-88a976eaf8d6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:37.895984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef6a08-1b75-4f5a-948c-f2465b1a035b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:37.896644 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef6a08-1b75-4f5a-948c-f2465b1a035b admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:37.896955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef6a08-1b75-4f5a-948c-f2465b1a035b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:37.897239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef6a08-1b75-4f5a-948c-f2465b1a035b admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:37.897992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef6a08-1b75-4f5a-948c-f2465b1a035b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:37.899993 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef6a08-1b75-4f5a-948c-f2465b1a035b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:37.932058 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 910/1816] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:37 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:37.936132 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad376a8-9ebc-4c6e-8e44-024651942797 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:37.948180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad376a8-9ebc-4c6e-8e44-024651942797 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:38.016639 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f59812-bd59-4007-a7f6-d84456b3955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:38.017294 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f59812-bd59-4007-a7f6-d84456b3955d admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:38.017602 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f59812-bd59-4007-a7f6-d84456b3955d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:38.017888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f59812-bd59-4007-a7f6-d84456b3955d admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:38.018564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f59812-bd59-4007-a7f6-d84456b3955d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:38.022941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f59812-bd59-4007-a7f6-d84456b3955d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:38.057820 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 907/1817] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:37 2022] POST /identity/v3/users => generated 300 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:38.059990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef6a08-1b75-4f5a-948c-f2465b1a035b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:38.078894 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef6a08-1b75-4f5a-948c-f2465b1a035b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:38.088336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b3dfc-7871-4153-94ca-4ba21283b42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:38.088980 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b3dfc-7871-4153-94ca-4ba21283b42a admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:38.089425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b3dfc-7871-4153-94ca-4ba21283b42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:38.089801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b3dfc-7871-4153-94ca-4ba21283b42a admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:38.091359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b3dfc-7871-4153-94ca-4ba21283b42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:38.093215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b3dfc-7871-4153-94ca-4ba21283b42a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:38.119044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f59812-bd59-4007-a7f6-d84456b3955d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:38.119307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 911/1818] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:44:38.149555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f59812-bd59-4007-a7f6-d84456b3955d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:38.167023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d96a5-6ffd-41cd-b025-8b14b40b9446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:38.168493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d96a5-6ffd-41cd-b025-8b14b40b9446 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:38.169211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d96a5-6ffd-41cd-b025-8b14b40b9446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:38.172210 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d96a5-6ffd-41cd-b025-8b14b40b9446 admin admin] PATH_INFO: `/v3/projects/8961e2009b764b4eb1c2ce5c162afae0/users/dbea2a62810648f586cb1690f0a75208/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:38.225569 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d96a5-6ffd-41cd-b025-8b14b40b9446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8961e2009b764b4eb1c2ce5c162afae0, user_id=dbea2a62810648f586cb1690f0a75208, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:38.237070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d96a5-6ffd-41cd-b025-8b14b40b9446 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:38.277675 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 908/1819] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:38 2022] PUT /identity/v3/projects/8961e2009b764b4eb1c2ce5c162afae0/users/dbea2a62810648f586cb1690f0a75208/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:38.281685 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b3dfc-7871-4153-94ca-4ba21283b42a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:38.299537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b3dfc-7871-4153-94ca-4ba21283b42a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:38.470342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4fdd3-81b0-4ade-8b0d-bff43dc5bfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:38.470995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4fdd3-81b0-4ade-8b0d-bff43dc5bfb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:38.471818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4fdd3-81b0-4ade-8b0d-bff43dc5bfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:38.472117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4fdd3-81b0-4ade-8b0d-bff43dc5bfb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:38.472883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d4fdd3-81b0-4ade-8b0d-bff43dc5bfb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:38.475918 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d4fdd3-81b0-4ade-8b0d-bff43dc5bfb0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:38.503521 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d96a5-6ffd-41cd-b025-8b14b40b9446 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:38.509930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 912/1820] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:38.516437 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d96a5-6ffd-41cd-b025-8b14b40b9446 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:38.523209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b522a-f4bf-47a2-a61f-27031991d53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:38.524197 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b522a-f4bf-47a2-a61f-27031991d53a admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:38.524346 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b522a-f4bf-47a2-a61f-27031991d53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:38.524586 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b522a-f4bf-47a2-a61f-27031991d53a admin admin] PATH_INFO: `/v3/projects/8961e2009b764b4eb1c2ce5c162afae0/users/dbea2a62810648f586cb1690f0a75208/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:38.526981 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b522a-f4bf-47a2-a61f-27031991d53a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8961e2009b764b4eb1c2ce5c162afae0, user_id=dbea2a62810648f586cb1690f0a75208, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:38.530274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b522a-f4bf-47a2-a61f-27031991d53a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:38.567165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 909/1821] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:38 2022] PUT /identity/v3/projects/8961e2009b764b4eb1c2ce5c162afae0/users/dbea2a62810648f586cb1690f0a75208/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:38.575018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a007b-f483-478a-bc03-fcb7cc6f691d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:38.576631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a007b-f483-478a-bc03-fcb7cc6f691d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:38.581446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a007b-f483-478a-bc03-fcb7cc6f691d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:38.693879 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8a6a007b-f483-478a-bc03-fcb7cc6f691d None None] MFA Rules not processed for user `dbea2a62810648f586cb1690f0a75208`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:38.697560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8a6a007b-f483-478a-bc03-fcb7cc6f691d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:38.904577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 913/1822] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:38.934935 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b522a-f4bf-47a2-a61f-27031991d53a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:38.936787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-962b522a-f4bf-47a2-a61f-27031991d53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:39.028576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:39.069315 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b522a-f4bf-47a2-a61f-27031991d53a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:39.080153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f82660-838c-46f9-85b8-e7ff9f2160f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:39.080766 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f82660-838c-46f9-85b8-e7ff9f2160f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:39.081049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f82660-838c-46f9-85b8-e7ff9f2160f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:39.084018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f82660-838c-46f9-85b8-e7ff9f2160f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:39.108223 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f82660-838c-46f9-85b8-e7ff9f2160f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:39.110731 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f82660-838c-46f9-85b8-e7ff9f2160f6 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:39.128247 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 910/1823] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:39.262359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a007b-f483-478a-bc03-fcb7cc6f691d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:39.265015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8a6a007b-f483-478a-bc03-fcb7cc6f691d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:39.271396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f82660-838c-46f9-85b8-e7ff9f2160f6 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:39.341232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:39.371201 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a007b-f483-478a-bc03-fcb7cc6f691d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:39.377453 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e768fe-84c1-4023-b313-d560b4bb3746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:39.378256 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e768fe-84c1-4023-b313-d560b4bb3746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:39.378535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e768fe-84c1-4023-b313-d560b4bb3746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:39.381417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e768fe-84c1-4023-b313-d560b4bb3746 admin admin] PATH_INFO: `/v3/users/f929ee4769cb46c5908c9b125be801c2` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:39.384164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e768fe-84c1-4023-b313-d560b4bb3746 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f929ee4769cb46c5908c9b125be801c2)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:39.388193 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e768fe-84c1-4023-b313-d560b4bb3746 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:39.435783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f82660-838c-46f9-85b8-e7ff9f2160f6 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:39.455299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b461d-f4b3-41f5-b1cf-863c0758c7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:39.456085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b461d-f4b3-41f5-b1cf-863c0758c7c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:39.456374 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b461d-f4b3-41f5-b1cf-863c0758c7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:39.459181 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b461d-f4b3-41f5-b1cf-863c0758c7c0 admin admin] PATH_INFO: `/v3/users/649d80caa93b48e9b9d989a31b5d476c` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:39.468273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b461d-f4b3-41f5-b1cf-863c0758c7c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=649d80caa93b48e9b9d989a31b5d476c)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:39.471625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b461d-f4b3-41f5-b1cf-863c0758c7c0 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:39.509320 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a2e768fe-84c1-4023-b313-d560b4bb3746 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f929ee4769cb46c5908c9b125be801c2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:39.540061 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a2e768fe-84c1-4023-b313-d560b4bb3746 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f929ee4769cb46c5908c9b125be801c2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:39.541554 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a2e768fe-84c1-4023-b313-d560b4bb3746 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f929ee4769cb46c5908c9b125be801c2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:39.557801 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a2e768fe-84c1-4023-b313-d560b4bb3746 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f929ee4769cb46c5908c9b125be801c2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:39.577664 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a2e768fe-84c1-4023-b313-d560b4bb3746 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f929ee4769cb46c5908c9b125be801c2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:39.596059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 914/1824] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:39 2022] DELETE /identity/v3/users/f929ee4769cb46c5908c9b125be801c2 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:39.605871 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685fe9a-6db6-4ab6-b4fd-08c425e26dc5 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:39.606741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685fe9a-6db6-4ab6-b4fd-08c425e26dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:39.608214 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685fe9a-6db6-4ab6-b4fd-08c425e26dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:39.619060 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f67b461d-f4b3-41f5-b1cf-863c0758c7c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '649d80caa93b48e9b9d989a31b5d476c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:39.636079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f67b461d-f4b3-41f5-b1cf-863c0758c7c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '649d80caa93b48e9b9d989a31b5d476c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:39.657259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2685fe9a-6db6-4ab6-b4fd-08c425e26dc5 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:39.659175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2685fe9a-6db6-4ab6-b4fd-08c425e26dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:39.661687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f67b461d-f4b3-41f5-b1cf-863c0758c7c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '649d80caa93b48e9b9d989a31b5d476c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:39.662676 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f67b461d-f4b3-41f5-b1cf-863c0758c7c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '649d80caa93b48e9b9d989a31b5d476c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:39.665099 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f67b461d-f4b3-41f5-b1cf-863c0758c7c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '649d80caa93b48e9b9d989a31b5d476c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:39.716784 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 911/1825] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:39 2022] DELETE /identity/v3/users/649d80caa93b48e9b9d989a31b5d476c => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:39.719422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95949458-5ec8-4170-b555-3384f2ab9f9d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:39.719844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95949458-5ec8-4170-b555-3384f2ab9f9d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:39.720255 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95949458-5ec8-4170-b555-3384f2ab9f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:39.753315 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-95949458-5ec8-4170-b555-3384f2ab9f9d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:39.755549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-95949458-5ec8-4170-b555-3384f2ab9f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:39.843087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 915/1826] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:39.859633 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:39.885333 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 912/1827] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:39.920958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685fe9a-6db6-4ab6-b4fd-08c425e26dc5 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:39.924710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2685fe9a-6db6-4ab6-b4fd-08c425e26dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:39.971115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95949458-5ec8-4170-b555-3384f2ab9f9d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:39.973824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-95949458-5ec8-4170-b555-3384f2ab9f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:40.109134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685fe9a-6db6-4ab6-b4fd-08c425e26dc5 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:40.145492 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95949458-5ec8-4170-b555-3384f2ab9f9d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:40.155078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6fe3c-b3f0-4f18-a006-9a1a8648d8b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:40.155754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6fe3c-b3f0-4f18-a006-9a1a8648d8b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:40.156044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6fe3c-b3f0-4f18-a006-9a1a8648d8b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:40.156479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6fe3c-b3f0-4f18-a006-9a1a8648d8b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:40.179078 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:40.191014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6fe3c-b3f0-4f18-a006-9a1a8648d8b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:40.192818 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6fe3c-b3f0-4f18-a006-9a1a8648d8b1 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:40.220070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 913/1828] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:40.288885 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e3eaa-b2e2-4db8-ae49-ce31c2b7cc0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:40.289373 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e3eaa-b2e2-4db8-ae49-ce31c2b7cc0f service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:40.289684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e3eaa-b2e2-4db8-ae49-ce31c2b7cc0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:40.290109 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e3eaa-b2e2-4db8-ae49-ce31c2b7cc0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:40.313116 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e3eaa-b2e2-4db8-ae49-ce31c2b7cc0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:40.314651 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e3eaa-b2e2-4db8-ae49-ce31c2b7cc0f service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:40.345953 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 916/1829] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:41.880641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6fe3c-b3f0-4f18-a006-9a1a8648d8b1 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:41.883895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7db6fe3c-b3f0-4f18-a006-9a1a8648d8b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:41.901267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6fe3c-b3f0-4f18-a006-9a1a8648d8b1 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:41.906103 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f9da2-0e5a-4a59-9d88-13ac40f6793f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:41.906787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f9da2-0e5a-4a59-9d88-13ac40f6793f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:41.907065 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f9da2-0e5a-4a59-9d88-13ac40f6793f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:41.907343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f9da2-0e5a-4a59-9d88-13ac40f6793f admin admin] PATH_INFO: `/v3/projects/4af1e50bdd9849f9b0066f134623237b` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:41.908617 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f9da2-0e5a-4a59-9d88-13ac40f6793f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4af1e50bdd9849f9b0066f134623237b)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:41.910884 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f9da2-0e5a-4a59-9d88-13ac40f6793f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:42.024203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e3eaa-b2e2-4db8-ae49-ce31c2b7cc0f service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:42.027806 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-061e3eaa-b2e2-4db8-ae49-ce31c2b7cc0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:42.043445 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-079f9da2-0e5a-4a59-9d88-13ac40f6793f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4af1e50bdd9849f9b0066f134623237b', 'request_id': 'req-079f9da2-0e5a-4a59-9d88-13ac40f6793f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.054407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-079f9da2-0e5a-4a59-9d88-13ac40f6793f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4af1e50bdd9849f9b0066f134623237b', 'request_id': 'req-079f9da2-0e5a-4a59-9d88-13ac40f6793f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.063325 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e3eaa-b2e2-4db8-ae49-ce31c2b7cc0f service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:42.072205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-079f9da2-0e5a-4a59-9d88-13ac40f6793f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4af1e50bdd9849f9b0066f134623237b', 'request_id': 'req-079f9da2-0e5a-4a59-9d88-13ac40f6793f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.099180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-079f9da2-0e5a-4a59-9d88-13ac40f6793f admin admin] The token cache is being invalidate because project 4af1e50bdd9849f9b0066f134623237b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:42.099472 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-079f9da2-0e5a-4a59-9d88-13ac40f6793f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.100223 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-079f9da2-0e5a-4a59-9d88-13ac40f6793f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.105437 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 914/1830] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:41 2022] DELETE /identity/v3/projects/4af1e50bdd9849f9b0066f134623237b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:42.106488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f9da2-0e5a-4a59-9d88-13ac40f6793f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:42.108286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-079f9da2-0e5a-4a59-9d88-13ac40f6793f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:42.204322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c29a8-74d7-42ad-aed3-c73492ba350e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:42.205012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c29a8-74d7-42ad-aed3-c73492ba350e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:42.205301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c29a8-74d7-42ad-aed3-c73492ba350e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:42.205706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c29a8-74d7-42ad-aed3-c73492ba350e admin admin] PATH_INFO: `/v3/projects/4a95dcf20958490aa6ffaf48af355f1f` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:42.207717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c29a8-74d7-42ad-aed3-c73492ba350e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a95dcf20958490aa6ffaf48af355f1f)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:42.209907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c29a8-74d7-42ad-aed3-c73492ba350e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:42.276340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f57c29a8-74d7-42ad-aed3-c73492ba350e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a95dcf20958490aa6ffaf48af355f1f', 'request_id': 'req-f57c29a8-74d7-42ad-aed3-c73492ba350e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.277541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f9da2-0e5a-4a59-9d88-13ac40f6793f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:42.289753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f57c29a8-74d7-42ad-aed3-c73492ba350e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a95dcf20958490aa6ffaf48af355f1f', 'request_id': 'req-f57c29a8-74d7-42ad-aed3-c73492ba350e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.303627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f57c29a8-74d7-42ad-aed3-c73492ba350e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a95dcf20958490aa6ffaf48af355f1f', 'request_id': 'req-f57c29a8-74d7-42ad-aed3-c73492ba350e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.318914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f57c29a8-74d7-42ad-aed3-c73492ba350e admin admin] The token cache is being invalidate because project 4a95dcf20958490aa6ffaf48af355f1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:42.319164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f57c29a8-74d7-42ad-aed3-c73492ba350e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.319856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f57c29a8-74d7-42ad-aed3-c73492ba350e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.321978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 917/1831] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:42 2022] DELETE /identity/v3/projects/4a95dcf20958490aa6ffaf48af355f1f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:44:42.328984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c29a8-74d7-42ad-aed3-c73492ba350e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:42.331676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f57c29a8-74d7-42ad-aed3-c73492ba350e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:42.334247 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:42.408186 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf5cec-48bc-4b8f-8ca1-7ba1fa656edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:42.409490 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf5cec-48bc-4b8f-8ca1-7ba1fa656edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:42.410120 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf5cec-48bc-4b8f-8ca1-7ba1fa656edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:42.410596 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf5cec-48bc-4b8f-8ca1-7ba1fa656edc admin admin] PATH_INFO: `/v3/users/dbea2a62810648f586cb1690f0a75208` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:42.413242 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf5cec-48bc-4b8f-8ca1-7ba1fa656edc admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbea2a62810648f586cb1690f0a75208)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:42.420271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf5cec-48bc-4b8f-8ca1-7ba1fa656edc admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:42.473744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:42.534433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c29a8-74d7-42ad-aed3-c73492ba350e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:42.566018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-34bf5cec-48bc-4b8f-8ca1-7ba1fa656edc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbea2a62810648f586cb1690f0a75208'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.572045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08702cc8-9c58-49cf-bbe7-e35eab7ae716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:42.575261 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08702cc8-9c58-49cf-bbe7-e35eab7ae716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:42.575535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08702cc8-9c58-49cf-bbe7-e35eab7ae716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:42.577668 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08702cc8-9c58-49cf-bbe7-e35eab7ae716 admin admin] PATH_INFO: `/v3/users/8f6bbd0eaa654f349f3f1e5cdc62c35b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:42.581551 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08702cc8-9c58-49cf-bbe7-e35eab7ae716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f6bbd0eaa654f349f3f1e5cdc62c35b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:42.586871 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08702cc8-9c58-49cf-bbe7-e35eab7ae716 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:42.608462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-34bf5cec-48bc-4b8f-8ca1-7ba1fa656edc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbea2a62810648f586cb1690f0a75208'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.635612 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-34bf5cec-48bc-4b8f-8ca1-7ba1fa656edc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbea2a62810648f586cb1690f0a75208'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.637644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-34bf5cec-48bc-4b8f-8ca1-7ba1fa656edc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbea2a62810648f586cb1690f0a75208'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.639546 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-34bf5cec-48bc-4b8f-8ca1-7ba1fa656edc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbea2a62810648f586cb1690f0a75208'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.726404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 915/1832] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:42 2022] DELETE /identity/v3/users/dbea2a62810648f586cb1690f0a75208 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:42.791547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-08702cc8-9c58-49cf-bbe7-e35eab7ae716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f6bbd0eaa654f349f3f1e5cdc62c35b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.858969 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-08702cc8-9c58-49cf-bbe7-e35eab7ae716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f6bbd0eaa654f349f3f1e5cdc62c35b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.860389 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-08702cc8-9c58-49cf-bbe7-e35eab7ae716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f6bbd0eaa654f349f3f1e5cdc62c35b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.885978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-08702cc8-9c58-49cf-bbe7-e35eab7ae716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f6bbd0eaa654f349f3f1e5cdc62c35b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.908168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-08702cc8-9c58-49cf-bbe7-e35eab7ae716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f6bbd0eaa654f349f3f1e5cdc62c35b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:42.926425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 918/1833] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:42 2022] DELETE /identity/v3/users/8f6bbd0eaa654f349f3f1e5cdc62c35b => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:43.130360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf5cec-48bc-4b8f-8ca1-7ba1fa656edc admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:43.132960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-34bf5cec-48bc-4b8f-8ca1-7ba1fa656edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:43.178341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bb0e3-74d5-4f64-891c-a2a4d525e61f None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:43.178965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bb0e3-74d5-4f64-891c-a2a4d525e61f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:43.179348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bb0e3-74d5-4f64-891c-a2a4d525e61f None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:43.184222 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 919/1834] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 3 10:44:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:44:43.243665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d787bd-94b8-4dde-a7b9-d8cffbf82385 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:43.244722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d787bd-94b8-4dde-a7b9-d8cffbf82385 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:43.244722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d787bd-94b8-4dde-a7b9-d8cffbf82385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:43.325042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c4d787bd-94b8-4dde-a7b9-d8cffbf82385 None None] MFA Rules not processed for user `e66f25cc329f418eba6bd637bf3e8a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:43.327198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c4d787bd-94b8-4dde-a7b9-d8cffbf82385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:43.330400 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf5cec-48bc-4b8f-8ca1-7ba1fa656edc admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:43.461324 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:43.477274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e8c2d-8b76-4ac5-bac3-cfa5b0f256a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:43.478989 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e8c2d-8b76-4ac5-bac3-cfa5b0f256a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:43.479500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e8c2d-8b76-4ac5-bac3-cfa5b0f256a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:43.479980 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e8c2d-8b76-4ac5-bac3-cfa5b0f256a5 admin admin] PATH_INFO: `/v3/users/044d53fda6a54840a1d0563a96118622` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:43.483074 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e8c2d-8b76-4ac5-bac3-cfa5b0f256a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=044d53fda6a54840a1d0563a96118622)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:43.486105 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e8c2d-8b76-4ac5-bac3-cfa5b0f256a5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:43.562469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 920/1835] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 3 10:44:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:43.580500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ff8e8c2d-8b76-4ac5-bac3-cfa5b0f256a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '044d53fda6a54840a1d0563a96118622'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:43.617747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ff8e8c2d-8b76-4ac5-bac3-cfa5b0f256a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '044d53fda6a54840a1d0563a96118622'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:43.637663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ff8e8c2d-8b76-4ac5-bac3-cfa5b0f256a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '044d53fda6a54840a1d0563a96118622'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:43.638604 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ff8e8c2d-8b76-4ac5-bac3-cfa5b0f256a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '044d53fda6a54840a1d0563a96118622'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:43.639501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ff8e8c2d-8b76-4ac5-bac3-cfa5b0f256a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '044d53fda6a54840a1d0563a96118622'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:43.669373 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 916/1836] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:43 2022] DELETE /identity/v3/users/044d53fda6a54840a1d0563a96118622 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:43.676740 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff60718-f737-4e5a-8de2-b9f6e4999130 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:43.677258 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff60718-f737-4e5a-8de2-b9f6e4999130 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:43.677796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff60718-f737-4e5a-8de2-b9f6e4999130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:43.724671 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0ff60718-f737-4e5a-8de2-b9f6e4999130 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:43.727004 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0ff60718-f737-4e5a-8de2-b9f6e4999130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:43.872325 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 921/1837] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:43.897996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e8c2d-8b76-4ac5-bac3-cfa5b0f256a5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:43.899681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ff8e8c2d-8b76-4ac5-bac3-cfa5b0f256a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:43.923291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff60718-f737-4e5a-8de2-b9f6e4999130 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:43.927586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0ff60718-f737-4e5a-8de2-b9f6e4999130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:43.960884 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:43.975813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff60718-f737-4e5a-8de2-b9f6e4999130 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:43.997670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93230a-2e56-4cfa-9753-632c4bf65d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:43.999011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93230a-2e56-4cfa-9753-632c4bf65d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:43.999523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93230a-2e56-4cfa-9753-632c4bf65d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:44.000020 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93230a-2e56-4cfa-9753-632c4bf65d1d admin admin] PATH_INFO: `/v3/projects/8961e2009b764b4eb1c2ce5c162afae0` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:44.005160 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93230a-2e56-4cfa-9753-632c4bf65d1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8961e2009b764b4eb1c2ce5c162afae0)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:44.013027 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93230a-2e56-4cfa-9753-632c4bf65d1d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:44.063355 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e8c2d-8b76-4ac5-bac3-cfa5b0f256a5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:44.073784 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2810d-452b-4cb4-a351-15d6672b0b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:44.075940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2810d-452b-4cb4-a351-15d6672b0b46 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:44.076527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2810d-452b-4cb4-a351-15d6672b0b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:44.077027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2810d-452b-4cb4-a351-15d6672b0b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:44.103501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2810d-452b-4cb4-a351-15d6672b0b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:44.114718 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2810d-452b-4cb4-a351-15d6672b0b46 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:44.140137 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 917/1838] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:44.171917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0d93230a-2e56-4cfa-9753-632c4bf65d1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8961e2009b764b4eb1c2ce5c162afae0', 'request_id': 'req-0d93230a-2e56-4cfa-9753-632c4bf65d1d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:44.193140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0d93230a-2e56-4cfa-9753-632c4bf65d1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8961e2009b764b4eb1c2ce5c162afae0', 'request_id': 'req-0d93230a-2e56-4cfa-9753-632c4bf65d1d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:44.209070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0d93230a-2e56-4cfa-9753-632c4bf65d1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8961e2009b764b4eb1c2ce5c162afae0', 'request_id': 'req-0d93230a-2e56-4cfa-9753-632c4bf65d1d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:44.235750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0d93230a-2e56-4cfa-9753-632c4bf65d1d admin admin] The token cache is being invalidate because project 8961e2009b764b4eb1c2ce5c162afae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:44.236376 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0d93230a-2e56-4cfa-9753-632c4bf65d1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:44.237736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0d93230a-2e56-4cfa-9753-632c4bf65d1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:44.259288 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 922/1839] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:43 2022] DELETE /identity/v3/projects/8961e2009b764b4eb1c2ce5c162afae0 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:44.264846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b247c-6fb3-4f94-9e3b-a7c9a8145d9c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:44.266002 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b247c-6fb3-4f94-9e3b-a7c9a8145d9c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:44.266677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b247c-6fb3-4f94-9e3b-a7c9a8145d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:44.306333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93230a-2e56-4cfa-9753-632c4bf65d1d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:44.311864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0d93230a-2e56-4cfa-9753-632c4bf65d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:44.470396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-835b247c-6fb3-4f94-9e3b-a7c9a8145d9c None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:44.483785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-835b247c-6fb3-4f94-9e3b-a7c9a8145d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:44.651790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 918/1840] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:44.651790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b247c-6fb3-4f94-9e3b-a7c9a8145d9c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:44.655845 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-835b247c-6fb3-4f94-9e3b-a7c9a8145d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:44.663055 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93230a-2e56-4cfa-9753-632c4bf65d1d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:44.685678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1391e-57af-41ae-865e-e6b7e940baed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:44.692101 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1391e-57af-41ae-865e-e6b7e940baed service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:44.692589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1391e-57af-41ae-865e-e6b7e940baed service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:44.694275 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1391e-57af-41ae-865e-e6b7e940baed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:44.698756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b247c-6fb3-4f94-9e3b-a7c9a8145d9c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:44.704362 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4ff1391e-57af-41ae-865e-e6b7e940baed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:44.705867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62825e0-ba6d-47b7-97e7-d07aa59814bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:44.706652 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62825e0-ba6d-47b7-97e7-d07aa59814bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:44.707185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62825e0-ba6d-47b7-97e7-d07aa59814bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:44.707541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62825e0-ba6d-47b7-97e7-d07aa59814bc admin admin] PATH_INFO: `/v3/projects/bdf2b915055d47bdbc5f035fb161ac15` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:44.708916 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62825e0-ba6d-47b7-97e7-d07aa59814bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdf2b915055d47bdbc5f035fb161ac15)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:44.711021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62825e0-ba6d-47b7-97e7-d07aa59814bc admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:44.725929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:44.739500 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1391e-57af-41ae-865e-e6b7e940baed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:44.743491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1391e-57af-41ae-865e-e6b7e940baed service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:44.770111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 923/1841] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:44.773843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc01e4-df2d-4488-9e2a-92d94d6b651b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:44.774720 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc01e4-df2d-4488-9e2a-92d94d6b651b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:44.775883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc01e4-df2d-4488-9e2a-92d94d6b651b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:44.791442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a62825e0-ba6d-47b7-97e7-d07aa59814bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdf2b915055d47bdbc5f035fb161ac15', 'request_id': 'req-a62825e0-ba6d-47b7-97e7-d07aa59814bc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:44.800327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a62825e0-ba6d-47b7-97e7-d07aa59814bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdf2b915055d47bdbc5f035fb161ac15', 'request_id': 'req-a62825e0-ba6d-47b7-97e7-d07aa59814bc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:44.814966 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a62825e0-ba6d-47b7-97e7-d07aa59814bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdf2b915055d47bdbc5f035fb161ac15', 'request_id': 'req-a62825e0-ba6d-47b7-97e7-d07aa59814bc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:44.816710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-46cc01e4-df2d-4488-9e2a-92d94d6b651b None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:44.818743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-46cc01e4-df2d-4488-9e2a-92d94d6b651b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:44.845354 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a62825e0-ba6d-47b7-97e7-d07aa59814bc admin admin] The token cache is being invalidate because project bdf2b915055d47bdbc5f035fb161ac15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:44.845713 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a62825e0-ba6d-47b7-97e7-d07aa59814bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:44.847233 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a62825e0-ba6d-47b7-97e7-d07aa59814bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:44.863962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 919/1842] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:44 2022] DELETE /identity/v3/projects/bdf2b915055d47bdbc5f035fb161ac15 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:44.871390 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc57f7-5870-4eaa-a1d7-618bcaa0d2e9 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:44.871858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc57f7-5870-4eaa-a1d7-618bcaa0d2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:44.871994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc57f7-5870-4eaa-a1d7-618bcaa0d2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:44.945717 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-efdc57f7-5870-4eaa-a1d7-618bcaa0d2e9 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:44.952745 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-efdc57f7-5870-4eaa-a1d7-618bcaa0d2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:45.059773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 924/1843] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:45.064758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc01e4-df2d-4488-9e2a-92d94d6b651b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:45.084518 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc01e4-df2d-4488-9e2a-92d94d6b651b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:45.095560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95f73f-1da8-4ec4-bf10-9619db4a0b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:45.098514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95f73f-1da8-4ec4-bf10-9619db4a0b1d admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:45.099851 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95f73f-1da8-4ec4-bf10-9619db4a0b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:45.100293 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95f73f-1da8-4ec4-bf10-9619db4a0b1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:45.101251 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95f73f-1da8-4ec4-bf10-9619db4a0b1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:44:45.102580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95f73f-1da8-4ec4-bf10-9619db4a0b1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:45.106209 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95f73f-1da8-4ec4-bf10-9619db4a0b1d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:45.125132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:45.134755 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 920/1844] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:45.137508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc57f7-5870-4eaa-a1d7-618bcaa0d2e9 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:45.138146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 925/1845] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:44:45.148172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f649c07-fe60-4bc5-b664-c82a37f73963 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:45.148751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f649c07-fe60-4bc5-b664-c82a37f73963 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:45.149181 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f649c07-fe60-4bc5-b664-c82a37f73963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:45.165975 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc57f7-5870-4eaa-a1d7-618bcaa0d2e9 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:45.192170 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabd022-648f-43d5-9b61-2224c0b5ce53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:45.193239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabd022-648f-43d5-9b61-2224c0b5ce53 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:45.193834 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabd022-648f-43d5-9b61-2224c0b5ce53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:45.194319 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabd022-648f-43d5-9b61-2224c0b5ce53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:45.195197 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabd022-648f-43d5-9b61-2224c0b5ce53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:45.196327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7f649c07-fe60-4bc5-b664-c82a37f73963 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:45.197629 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7f649c07-fe60-4bc5-b664-c82a37f73963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:45.198399 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabd022-648f-43d5-9b61-2224c0b5ce53 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:45.212999 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 926/1846] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:45.214334 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f649c07-fe60-4bc5-b664-c82a37f73963 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:45.242385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f649c07-fe60-4bc5-b664-c82a37f73963 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:45.254260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173501b4-2aa2-4c4d-899c-bbe61b926331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:45.256685 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173501b4-2aa2-4c4d-899c-bbe61b926331 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:45.257752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173501b4-2aa2-4c4d-899c-bbe61b926331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:45.258540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173501b4-2aa2-4c4d-899c-bbe61b926331 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:45.268054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 921/1847] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:45 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:45.270040 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabd022-648f-43d5-9b61-2224c0b5ce53 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:45.273442 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173501b4-2aa2-4c4d-899c-bbe61b926331 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:44:45.274395 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173501b4-2aa2-4c4d-899c-bbe61b926331 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:45.278606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173501b4-2aa2-4c4d-899c-bbe61b926331 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:45.321779 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 927/1848] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:44:45.325504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173501b4-2aa2-4c4d-899c-bbe61b926331 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:45.368908 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173501b4-2aa2-4c4d-899c-bbe61b926331 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:45.428888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabd022-648f-43d5-9b61-2224c0b5ce53 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:45.437891 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2590ca-e65c-412c-9820-74abce56556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:45.437891 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2590ca-e65c-412c-9820-74abce56556d admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:45.437891 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2590ca-e65c-412c-9820-74abce56556d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:45.438315 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2590ca-e65c-412c-9820-74abce56556d admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:45.439097 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2590ca-e65c-412c-9820-74abce56556d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:45.444418 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2590ca-e65c-412c-9820-74abce56556d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:45.512528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 922/1849] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:45 2022] POST /identity/v3/users => generated 290 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:44:45.524068 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2590ca-e65c-412c-9820-74abce56556d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:45.569724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be59566-c990-4d6b-84eb-d4bd6cdbe693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:45.571054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be59566-c990-4d6b-84eb-d4bd6cdbe693 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:45.571360 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be59566-c990-4d6b-84eb-d4bd6cdbe693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:45.572333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be59566-c990-4d6b-84eb-d4bd6cdbe693 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:45.573042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be59566-c990-4d6b-84eb-d4bd6cdbe693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:45.578944 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be59566-c990-4d6b-84eb-d4bd6cdbe693 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:45.592816 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2590ca-e65c-412c-9820-74abce56556d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:45.609633 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef238a76-9478-46da-9b4f-e8a61b32e389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:45.610584 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef238a76-9478-46da-9b4f-e8a61b32e389 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:45.611343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef238a76-9478-46da-9b4f-e8a61b32e389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:45.611905 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef238a76-9478-46da-9b4f-e8a61b32e389 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:45.614399 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef238a76-9478-46da-9b4f-e8a61b32e389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:45.618727 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef238a76-9478-46da-9b4f-e8a61b32e389 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:45.622317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:45.672012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 923/1850] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:45.674642 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef238a76-9478-46da-9b4f-e8a61b32e389 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:45.683280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ef238a76-9478-46da-9b4f-e8a61b32e389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:45.710457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef238a76-9478-46da-9b4f-e8a61b32e389 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:45.718039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 928/1851] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:45 2022] POST /identity/v3/projects => generated 383 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:44:45.719534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be59566-c990-4d6b-84eb-d4bd6cdbe693 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:45.724422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da88f60-d790-4dea-b793-9fde1eb3523d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:45.726394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da88f60-d790-4dea-b793-9fde1eb3523d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:45.728014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da88f60-d790-4dea-b793-9fde1eb3523d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:45.728410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da88f60-d790-4dea-b793-9fde1eb3523d admin admin] PATH_INFO: `/v3/projects/00f8ce38df2a42b9885dd55e71dc9581` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:45.732283 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da88f60-d790-4dea-b793-9fde1eb3523d admin admin] RBAC: Authorizing `identity:delete_project(project_id=00f8ce38df2a42b9885dd55e71dc9581)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:45.740274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da88f60-d790-4dea-b793-9fde1eb3523d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:45.866375 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5da88f60-d790-4dea-b793-9fde1eb3523d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00f8ce38df2a42b9885dd55e71dc9581', 'request_id': 'req-5da88f60-d790-4dea-b793-9fde1eb3523d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:45.882968 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5da88f60-d790-4dea-b793-9fde1eb3523d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00f8ce38df2a42b9885dd55e71dc9581', 'request_id': 'req-5da88f60-d790-4dea-b793-9fde1eb3523d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:45.911194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5da88f60-d790-4dea-b793-9fde1eb3523d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00f8ce38df2a42b9885dd55e71dc9581', 'request_id': 'req-5da88f60-d790-4dea-b793-9fde1eb3523d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:45.936009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be59566-c990-4d6b-84eb-d4bd6cdbe693 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:45.940943 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5da88f60-d790-4dea-b793-9fde1eb3523d admin admin] The token cache is being invalidate because project 00f8ce38df2a42b9885dd55e71dc9581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:45.941346 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5da88f60-d790-4dea-b793-9fde1eb3523d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:45.944392 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5da88f60-d790-4dea-b793-9fde1eb3523d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:45.948385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 924/1852] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:45 2022] DELETE /identity/v3/projects/00f8ce38df2a42b9885dd55e71dc9581 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:45.950905 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da88f60-d790-4dea-b793-9fde1eb3523d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:45.957390 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5da88f60-d790-4dea-b793-9fde1eb3523d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:46.095525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52d161-b3df-4a86-bd5c-d8b5afe32ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:46.096999 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52d161-b3df-4a86-bd5c-d8b5afe32ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:46.098122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52d161-b3df-4a86-bd5c-d8b5afe32ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:46.098624 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52d161-b3df-4a86-bd5c-d8b5afe32ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:46.100670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52d161-b3df-4a86-bd5c-d8b5afe32ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:46.102133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da88f60-d790-4dea-b793-9fde1eb3523d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:46.105264 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52d161-b3df-4a86-bd5c-d8b5afe32ef5 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:46.108200 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880bee4-f20e-4c49-946b-8bf6fa8f2d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:46.108859 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880bee4-f20e-4c49-946b-8bf6fa8f2d0c admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:46.109143 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880bee4-f20e-4c49-946b-8bf6fa8f2d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:46.109406 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880bee4-f20e-4c49-946b-8bf6fa8f2d0c admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:46.110132 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880bee4-f20e-4c49-946b-8bf6fa8f2d0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:46.112165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880bee4-f20e-4c49-946b-8bf6fa8f2d0c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:46.138171 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 929/1853] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:46.138171 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52d161-b3df-4a86-bd5c-d8b5afe32ef5 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:46.142337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 925/1854] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:45 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:46.142594 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-dd52d161-b3df-4a86-bd5c-d8b5afe32ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:46.143770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880bee4-f20e-4c49-946b-8bf6fa8f2d0c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:46.146801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b880bee4-f20e-4c49-946b-8bf6fa8f2d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:46.161846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880bee4-f20e-4c49-946b-8bf6fa8f2d0c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:46.168255 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a9e80-140c-41b9-8659-59fe2e34cc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:46.169240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a9e80-140c-41b9-8659-59fe2e34cc94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:46.169240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a9e80-140c-41b9-8659-59fe2e34cc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:46.170197 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a9e80-140c-41b9-8659-59fe2e34cc94 admin admin] PATH_INFO: `/v3/projects/50faea3a8cee4d1089b561b2f2d8bdce/users/96a10bbafc0d4b51bd2a76c245a2849e/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:46.173912 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52d161-b3df-4a86-bd5c-d8b5afe32ef5 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:46.188278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960f5c6-3d06-46de-b585-f3703a889d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:46.190290 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960f5c6-3d06-46de-b585-f3703a889d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:46.190868 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960f5c6-3d06-46de-b585-f3703a889d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:46.191991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960f5c6-3d06-46de-b585-f3703a889d91 admin admin] PATH_INFO: `/v3/users/081b9ea718714c708df4f3662b7254e1` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:46.192275 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a9e80-140c-41b9-8659-59fe2e34cc94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50faea3a8cee4d1089b561b2f2d8bdce, user_id=96a10bbafc0d4b51bd2a76c245a2849e, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:46.194811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960f5c6-3d06-46de-b585-f3703a889d91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=081b9ea718714c708df4f3662b7254e1)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:46.195427 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a9e80-140c-41b9-8659-59fe2e34cc94 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:46.199327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960f5c6-3d06-46de-b585-f3703a889d91 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:46.205381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:46.218481 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 926/1855] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:46 2022] PUT /identity/v3/projects/50faea3a8cee4d1089b561b2f2d8bdce/users/96a10bbafc0d4b51bd2a76c245a2849e/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:46.219842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a9e80-140c-41b9-8659-59fe2e34cc94 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:46.307255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5960f5c6-3d06-46de-b585-f3703a889d91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '081b9ea718714c708df4f3662b7254e1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:46.312979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:46.342622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5960f5c6-3d06-46de-b585-f3703a889d91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '081b9ea718714c708df4f3662b7254e1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:46.343916 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5960f5c6-3d06-46de-b585-f3703a889d91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '081b9ea718714c708df4f3662b7254e1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:46.361147 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a9e80-140c-41b9-8659-59fe2e34cc94 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:46.363527 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5960f5c6-3d06-46de-b585-f3703a889d91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '081b9ea718714c708df4f3662b7254e1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:46.399223 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeac6b-5d63-4089-9336-1d44103a8d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:46.400722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeac6b-5d63-4089-9336-1d44103a8d10 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:46.401410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeac6b-5d63-4089-9336-1d44103a8d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:46.401780 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5960f5c6-3d06-46de-b585-f3703a889d91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '081b9ea718714c708df4f3662b7254e1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:46.402324 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeac6b-5d63-4089-9336-1d44103a8d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:46.403889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbeac6b-5d63-4089-9336-1d44103a8d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:46.412279 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbeac6b-5d63-4089-9336-1d44103a8d10 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:46.416933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 930/1856] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:46 2022] DELETE /identity/v3/users/081b9ea718714c708df4f3662b7254e1 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:46.429970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7efe5-c22c-4194-b7e5-68399910c0f0 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:46.431187 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7efe5-c22c-4194-b7e5-68399910c0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:46.432018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7efe5-c22c-4194-b7e5-68399910c0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:46.450804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 927/1857] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:46.457981 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeac6b-5d63-4089-9336-1d44103a8d10 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:46.460540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5cbeac6b-5d63-4089-9336-1d44103a8d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:46.479037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeac6b-5d63-4089-9336-1d44103a8d10 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:46.505581 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-cbd7efe5-c22c-4194-b7e5-68399910c0f0 None None] MFA Rules not processed for user `96a10bbafc0d4b51bd2a76c245a2849e`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:46.508031 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-cbd7efe5-c22c-4194-b7e5-68399910c0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:46.622954 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad5d98-8960-4f08-96b6-2c923d186394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:46.623923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 931/1858] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:46.624944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad5d98-8960-4f08-96b6-2c923d186394 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:46.625506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad5d98-8960-4f08-96b6-2c923d186394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:46.625992 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad5d98-8960-4f08-96b6-2c923d186394 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:46.627837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad5d98-8960-4f08-96b6-2c923d186394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:46.630236 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7efe5-c22c-4194-b7e5-68399910c0f0 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:46.632803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad5d98-8960-4f08-96b6-2c923d186394 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:46.634377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-cbd7efe5-c22c-4194-b7e5-68399910c0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:46.667364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 928/1859] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:46.668281 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad5d98-8960-4f08-96b6-2c923d186394 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:46.694212 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad5d98-8960-4f08-96b6-2c923d186394 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:46.695555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7efe5-c22c-4194-b7e5-68399910c0f0 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:46.703490 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82ba38-1ae6-4b11-967b-10cb38b384f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:46.704208 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82ba38-1ae6-4b11-967b-10cb38b384f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:46.704489 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82ba38-1ae6-4b11-967b-10cb38b384f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:46.704768 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82ba38-1ae6-4b11-967b-10cb38b384f2 admin admin] PATH_INFO: `/v3/projects/6db800219dfe4b508646e0157f584a74/users/6409cf1abe914d3e8aebbde25fda6582/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:46.706028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ad47e-d1ad-4c4b-bf6e-eea933a6cdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:46.708415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ad47e-d1ad-4c4b-bf6e-eea933a6cdc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:46.710258 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ad47e-d1ad-4c4b-bf6e-eea933a6cdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:46.710988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ad47e-d1ad-4c4b-bf6e-eea933a6cdc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:46.712536 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ad47e-d1ad-4c4b-bf6e-eea933a6cdc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:46.716840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ad47e-d1ad-4c4b-bf6e-eea933a6cdc1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:46.734061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82ba38-1ae6-4b11-967b-10cb38b384f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db800219dfe4b508646e0157f584a74, user_id=6409cf1abe914d3e8aebbde25fda6582, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:46.736906 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82ba38-1ae6-4b11-967b-10cb38b384f2 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:46.760313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 929/1860] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:46 2022] PUT /identity/v3/projects/6db800219dfe4b508646e0157f584a74/users/6409cf1abe914d3e8aebbde25fda6582/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:46.774490 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95892be-40ae-41b4-8847-c15068d6fe2c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:46.774889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95892be-40ae-41b4-8847-c15068d6fe2c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:46.775264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95892be-40ae-41b4-8847-c15068d6fe2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:46.811901 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 932/1861] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:46 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:44:46.817160 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ad47e-d1ad-4c4b-bf6e-eea933a6cdc1 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:46.827651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e95892be-40ae-41b4-8847-c15068d6fe2c None None] MFA Rules not processed for user `6409cf1abe914d3e8aebbde25fda6582`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:46.829035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e95892be-40ae-41b4-8847-c15068d6fe2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:46.934697 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:46.941857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 930/1862] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:46.958505 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95892be-40ae-41b4-8847-c15068d6fe2c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:46.960095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e95892be-40ae-41b4-8847-c15068d6fe2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:46.972261 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ad47e-d1ad-4c4b-bf6e-eea933a6cdc1 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:46.992151 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac9976-046d-4145-86d6-799ed506cba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:46.993513 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac9976-046d-4145-86d6-799ed506cba6 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:46.993787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac9976-046d-4145-86d6-799ed506cba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:46.994173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac9976-046d-4145-86d6-799ed506cba6 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:46.994974 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac9976-046d-4145-86d6-799ed506cba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:46.997260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac9976-046d-4145-86d6-799ed506cba6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:47.066959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 933/1863] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:46 2022] POST /identity/v3/users => generated 296 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:47.066959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac9976-046d-4145-86d6-799ed506cba6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:47.077189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-dcac9976-046d-4145-86d6-799ed506cba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:47.094902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:47.114791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac9976-046d-4145-86d6-799ed506cba6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:47.130595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95892be-40ae-41b4-8847-c15068d6fe2c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:47.134317 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8107c-9432-4ae9-b940-049cc6899f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:47.134980 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8107c-9432-4ae9-b940-049cc6899f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:47.135304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8107c-9432-4ae9-b940-049cc6899f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:47.135539 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8107c-9432-4ae9-b940-049cc6899f74 admin admin] PATH_INFO: `/v3/projects/98164a766974409198b55e0b754aff27` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:47.136929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8107c-9432-4ae9-b940-049cc6899f74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98164a766974409198b55e0b754aff27)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:47.139136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8107c-9432-4ae9-b940-049cc6899f74 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:47.153646 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03966dd1-10dd-4cd6-b8cc-feb5374ec840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:47.154989 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03966dd1-10dd-4cd6-b8cc-feb5374ec840 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:47.155569 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03966dd1-10dd-4cd6-b8cc-feb5374ec840 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:47.156159 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03966dd1-10dd-4cd6-b8cc-feb5374ec840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:47.185657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03966dd1-10dd-4cd6-b8cc-feb5374ec840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:47.187856 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03966dd1-10dd-4cd6-b8cc-feb5374ec840 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:47.201284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 931/1864] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:44:47.202792 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03966dd1-10dd-4cd6-b8cc-feb5374ec840 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:47.215132 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03966dd1-10dd-4cd6-b8cc-feb5374ec840 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:47.220631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6981eb-620e-4bfc-ba1a-b25371714d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:47.221387 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6981eb-620e-4bfc-ba1a-b25371714d56 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:47.221781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6981eb-620e-4bfc-ba1a-b25371714d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:47.222140 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6981eb-620e-4bfc-ba1a-b25371714d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:47.223194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6981eb-620e-4bfc-ba1a-b25371714d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:47.225511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6981eb-620e-4bfc-ba1a-b25371714d56 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:47.238134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 932/1865] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:47.242488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6981eb-620e-4bfc-ba1a-b25371714d56 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:47.260331 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6981eb-620e-4bfc-ba1a-b25371714d56 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:47.263973 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d5e8107c-9432-4ae9-b940-049cc6899f74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98164a766974409198b55e0b754aff27', 'request_id': 'req-d5e8107c-9432-4ae9-b940-049cc6899f74'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:47.280110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d5e8107c-9432-4ae9-b940-049cc6899f74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98164a766974409198b55e0b754aff27', 'request_id': 'req-d5e8107c-9432-4ae9-b940-049cc6899f74'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:47.293307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d5e8107c-9432-4ae9-b940-049cc6899f74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98164a766974409198b55e0b754aff27', 'request_id': 'req-d5e8107c-9432-4ae9-b940-049cc6899f74'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:47.318073 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d5e8107c-9432-4ae9-b940-049cc6899f74 admin admin] The token cache is being invalidate because project 98164a766974409198b55e0b754aff27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:47.318899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d5e8107c-9432-4ae9-b940-049cc6899f74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:47.320414 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d5e8107c-9432-4ae9-b940-049cc6899f74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:47.333255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 934/1866] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:47 2022] DELETE /identity/v3/projects/98164a766974409198b55e0b754aff27 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:47.340793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543823f7-b7c1-4a99-aa60-80f1cc31865a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:47.341800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543823f7-b7c1-4a99-aa60-80f1cc31865a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:47.342764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543823f7-b7c1-4a99-aa60-80f1cc31865a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:47.401918 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-543823f7-b7c1-4a99-aa60-80f1cc31865a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:47.403698 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bc523-f841-4600-8376-3171889a46ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:47.404168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-543823f7-b7c1-4a99-aa60-80f1cc31865a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:47.406154 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bc523-f841-4600-8376-3171889a46ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:47.407183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bc523-f841-4600-8376-3171889a46ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:47.407759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bc523-f841-4600-8376-3171889a46ac admin admin] PATH_INFO: `/v3/projects/f60008a12be54711b174a928be3dfc03/users/f1b63970885549f1832fd7ff51d52ea4/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:47.429716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 935/1867] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:47.444777 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790feda8-b7a8-423c-b433-013d14c2eae0 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:47.450214 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790feda8-b7a8-423c-b433-013d14c2eae0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:47.451099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790feda8-b7a8-423c-b433-013d14c2eae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:47.494600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bc523-f841-4600-8376-3171889a46ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f60008a12be54711b174a928be3dfc03, user_id=f1b63970885549f1832fd7ff51d52ea4, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:47.504787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-790feda8-b7a8-423c-b433-013d14c2eae0 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:47.508754 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-790feda8-b7a8-423c-b433-013d14c2eae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:47.524974 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bc523-f841-4600-8376-3171889a46ac admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:47.526141 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 936/1868] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:47.531549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790feda8-b7a8-423c-b433-013d14c2eae0 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:47.556213 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790feda8-b7a8-423c-b433-013d14c2eae0 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:47.569657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba87c9-2e67-4a1a-9346-7757cc1a30f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:47.571637 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba87c9-2e67-4a1a-9346-7757cc1a30f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:47.572519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba87c9-2e67-4a1a-9346-7757cc1a30f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:47.573720 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba87c9-2e67-4a1a-9346-7757cc1a30f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:47.575311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba87c9-2e67-4a1a-9346-7757cc1a30f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:44:47.575962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba87c9-2e67-4a1a-9346-7757cc1a30f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:47.579102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba87c9-2e67-4a1a-9346-7757cc1a30f4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:47.604066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 933/1869] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:47 2022] PUT /identity/v3/projects/f60008a12be54711b174a928be3dfc03/users/f1b63970885549f1832fd7ff51d52ea4/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:47.604311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 937/1870] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:44:47.606142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba87c9-2e67-4a1a-9346-7757cc1a30f4 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:47.606893 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bc523-f841-4600-8376-3171889a46ac admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:47.610676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-60ba87c9-2e67-4a1a-9346-7757cc1a30f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:47.639005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba87c9-2e67-4a1a-9346-7757cc1a30f4 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:47.807828 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bc523-f841-4600-8376-3171889a46ac admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:47.818235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3f387-8605-4b74-8eb9-da7888a3a620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:47.820433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3f387-8605-4b74-8eb9-da7888a3a620 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:47.820822 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3f387-8605-4b74-8eb9-da7888a3a620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:47.821111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3f387-8605-4b74-8eb9-da7888a3a620 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:47.825941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3f387-8605-4b74-8eb9-da7888a3a620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:47.828333 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3f387-8605-4b74-8eb9-da7888a3a620 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:47.877828 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1e845-baf8-461b-bf71-f2096913fe20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:47.878484 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1e845-baf8-461b-bf71-f2096913fe20 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:47.878803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1e845-baf8-461b-bf71-f2096913fe20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:47.879084 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1e845-baf8-461b-bf71-f2096913fe20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:47.879810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1e845-baf8-461b-bf71-f2096913fe20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:47.881454 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1e845-baf8-461b-bf71-f2096913fe20 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:47.900302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 938/1871] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:47.902164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1e845-baf8-461b-bf71-f2096913fe20 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:47.914926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 934/1872] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:47 2022] POST /identity/v3/projects => generated 365 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:47.920378 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3f387-8605-4b74-8eb9-da7888a3a620 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:47.926255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1e845-baf8-461b-bf71-f2096913fe20 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:47.989199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:47.990111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:48.017847 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3f387-8605-4b74-8eb9-da7888a3a620 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:48.023998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe6871-4e06-49d3-801b-14be8d749148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:48.024650 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe6871-4e06-49d3-801b-14be8d749148 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:48.025081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe6871-4e06-49d3-801b-14be8d749148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:48.025361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe6871-4e06-49d3-801b-14be8d749148 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:48.026097 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe6871-4e06-49d3-801b-14be8d749148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:48.028096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe6871-4e06-49d3-801b-14be8d749148 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:48.050254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236ea50-73a0-4788-92fb-3ec91a950f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:48.052009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236ea50-73a0-4788-92fb-3ec91a950f6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:48.052526 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236ea50-73a0-4788-92fb-3ec91a950f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:48.053348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236ea50-73a0-4788-92fb-3ec91a950f6e admin admin] PATH_INFO: `/v3/projects/f60008a12be54711b174a928be3dfc03/users/f1b63970885549f1832fd7ff51d52ea4/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:48.058090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236ea50-73a0-4788-92fb-3ec91a950f6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f60008a12be54711b174a928be3dfc03, user_id=f1b63970885549f1832fd7ff51d52ea4, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:48.062606 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 935/1873] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:47 2022] POST /identity/v3/users => generated 283 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:44:48.066558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236ea50-73a0-4788-92fb-3ec91a950f6e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:48.068017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe6871-4e06-49d3-801b-14be8d749148 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:48.081288 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 939/1874] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:47 2022] PUT /identity/v3/projects/f60008a12be54711b174a928be3dfc03/users/f1b63970885549f1832fd7ff51d52ea4/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:48.081956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe6871-4e06-49d3-801b-14be8d749148 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:48.092419 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c60854-2cce-4e93-88d9-a254854c9a17 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:48.093052 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c60854-2cce-4e93-88d9-a254854c9a17 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:48.093633 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c60854-2cce-4e93-88d9-a254854c9a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:48.148601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d3c60854-2cce-4e93-88d9-a254854c9a17 None None] MFA Rules not processed for user `f1b63970885549f1832fd7ff51d52ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:48.149772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d3c60854-2cce-4e93-88d9-a254854c9a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:48.183696 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16565977-c97c-4729-9b4f-99a6ab3f0f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:48.184397 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16565977-c97c-4729-9b4f-99a6ab3f0f5c admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:48.184685 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16565977-c97c-4729-9b4f-99a6ab3f0f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:48.185016 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16565977-c97c-4729-9b4f-99a6ab3f0f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:48.186002 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16565977-c97c-4729-9b4f-99a6ab3f0f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:48.188035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16565977-c97c-4729-9b4f-99a6ab3f0f5c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:48.207843 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 936/1875] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:48.213118 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16565977-c97c-4729-9b4f-99a6ab3f0f5c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:48.256376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16565977-c97c-4729-9b4f-99a6ab3f0f5c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:48.271637 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66128e8c-692f-44d6-aa2a-bceaa0123e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:48.272310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66128e8c-692f-44d6-aa2a-bceaa0123e2f admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:48.274187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66128e8c-692f-44d6-aa2a-bceaa0123e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:48.274513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66128e8c-692f-44d6-aa2a-bceaa0123e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:48.275524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66128e8c-692f-44d6-aa2a-bceaa0123e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:48.281190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66128e8c-692f-44d6-aa2a-bceaa0123e2f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:48.308425 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 937/1876] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:48.309903 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66128e8c-692f-44d6-aa2a-bceaa0123e2f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:48.324720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66128e8c-692f-44d6-aa2a-bceaa0123e2f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:48.331501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4020d6e-2e2e-444b-a7e6-0656e70e9f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:48.332689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4020d6e-2e2e-444b-a7e6-0656e70e9f9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:48.332990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4020d6e-2e2e-444b-a7e6-0656e70e9f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:48.333279 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4020d6e-2e2e-444b-a7e6-0656e70e9f9f admin admin] PATH_INFO: `/v3/projects/9b28c0f9b2994d8d8f5efeb474e13777/users/8c13a30d8a7c447fb8e4ada755893d78/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:48.345770 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 940/1877] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:48.365022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c60854-2cce-4e93-88d9-a254854c9a17 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:48.372906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d3c60854-2cce-4e93-88d9-a254854c9a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:48.374090 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4020d6e-2e2e-444b-a7e6-0656e70e9f9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b28c0f9b2994d8d8f5efeb474e13777, user_id=8c13a30d8a7c447fb8e4ada755893d78, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:48.377095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4020d6e-2e2e-444b-a7e6-0656e70e9f9f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:48.399066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 938/1878] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:48 2022] PUT /identity/v3/projects/9b28c0f9b2994d8d8f5efeb474e13777/users/8c13a30d8a7c447fb8e4ada755893d78/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:44:48.414920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bb907-9338-4f48-95cb-9739d39eb000 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:48.415490 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bb907-9338-4f48-95cb-9739d39eb000 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:48.415953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bb907-9338-4f48-95cb-9739d39eb000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:48.454820 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-db7bb907-9338-4f48-95cb-9739d39eb000 None None] MFA Rules not processed for user `8c13a30d8a7c447fb8e4ada755893d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:48.456743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-db7bb907-9338-4f48-95cb-9739d39eb000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:48.524155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c60854-2cce-4e93-88d9-a254854c9a17 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:48.559903 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 939/1879] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:48.567917 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7bb907-9338-4f48-95cb-9739d39eb000 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:48.604047 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6358c-0b2d-4762-9192-723e834a9a5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:48.604703 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6358c-0b2d-4762-9192-723e834a9a5f service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:48.604986 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6358c-0b2d-4762-9192-723e834a9a5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:48.605261 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6358c-0b2d-4762-9192-723e834a9a5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:48.641508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:48.641776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:48.692534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6358c-0b2d-4762-9192-723e834a9a5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:48.694156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6358c-0b2d-4762-9192-723e834a9a5f service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:48.706474 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 941/1880] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:48.717288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7bb907-9338-4f48-95cb-9739d39eb000 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:48.723484 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29c23e-241e-4e1d-b349-d45df83e757a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:48.724019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29c23e-241e-4e1d-b349-d45df83e757a admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:48.724300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29c23e-241e-4e1d-b349-d45df83e757a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:48.724575 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29c23e-241e-4e1d-b349-d45df83e757a admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:48.725277 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29c23e-241e-4e1d-b349-d45df83e757a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:48.727285 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29c23e-241e-4e1d-b349-d45df83e757a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:48.810809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 940/1881] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:48 2022] POST /identity/v3/projects => generated 363 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:44:48.811452 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6358c-0b2d-4762-9192-723e834a9a5f service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:48.825903 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6358c-0b2d-4762-9192-723e834a9a5f service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:48.950142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f2a76-5e7e-4c72-9f01-25db6f01a7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:48.951825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f2a76-5e7e-4c72-9f01-25db6f01a7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:48.951825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f2a76-5e7e-4c72-9f01-25db6f01a7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:48.951825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f2a76-5e7e-4c72-9f01-25db6f01a7d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:48.965572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f2a76-5e7e-4c72-9f01-25db6f01a7d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:48.973015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f2a76-5e7e-4c72-9f01-25db6f01a7d5 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:49.081615 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 942/1882] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:48 2022] POST /identity/v3/users => generated 288 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:44:49.085885 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29c23e-241e-4e1d-b349-d45df83e757a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:49.103569 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29c23e-241e-4e1d-b349-d45df83e757a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:49.117940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda3a4b-e95f-4dad-bd66-f4976230cfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:49.118710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda3a4b-e95f-4dad-bd66-f4976230cfd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:49.119194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda3a4b-e95f-4dad-bd66-f4976230cfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:49.129045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda3a4b-e95f-4dad-bd66-f4976230cfd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:49.129846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda3a4b-e95f-4dad-bd66-f4976230cfd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:49.134306 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda3a4b-e95f-4dad-bd66-f4976230cfd9 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:49.192717 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 941/1883] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:49.195069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f2a76-5e7e-4c72-9f01-25db6f01a7d5 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:49.216148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f2a76-5e7e-4c72-9f01-25db6f01a7d5 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:49.234565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3836a8-70d0-48e0-a029-4476cd3b5917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:49.235152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3836a8-70d0-48e0-a029-4476cd3b5917 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:49.235448 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3836a8-70d0-48e0-a029-4476cd3b5917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:49.235728 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3836a8-70d0-48e0-a029-4476cd3b5917 admin admin] PATH_INFO: `/v3/projects/2404ff01f2f9416faa9910606af7bd87/users/853d107681d641ea95179e81d834faab/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:49.274277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3836a8-70d0-48e0-a029-4476cd3b5917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2404ff01f2f9416faa9910606af7bd87, user_id=853d107681d641ea95179e81d834faab, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:49.289388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3836a8-70d0-48e0-a029-4476cd3b5917 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:49.333150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 943/1884] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:49 2022] PUT /identity/v3/projects/2404ff01f2f9416faa9910606af7bd87/users/853d107681d641ea95179e81d834faab/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:49.334556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda3a4b-e95f-4dad-bd66-f4976230cfd9 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:49.365183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda3a4b-e95f-4dad-bd66-f4976230cfd9 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:49.447890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3836a8-70d0-48e0-a029-4476cd3b5917 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:49.558342 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d0866-cdd0-43fa-940e-73ed7a007c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:49.562070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d0866-cdd0-43fa-940e-73ed7a007c01 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:49.562070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d0866-cdd0-43fa-940e-73ed7a007c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:49.562070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d0866-cdd0-43fa-940e-73ed7a007c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:49.563702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d0866-cdd0-43fa-940e-73ed7a007c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:49.567932 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d0866-cdd0-43fa-940e-73ed7a007c01 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:49.591752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3836a8-70d0-48e0-a029-4476cd3b5917 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:49.598339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 942/1885] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:49.598485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420a8d4-2051-4ec2-88f9-629e0d87626d tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b63970885549f1832fd7ff51d52ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f60008a12be54711b174a928be3dfc03', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:49.598685 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420a8d4-2051-4ec2-88f9-629e0d87626d tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:49.598882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420a8d4-2051-4ec2-88f9-629e0d87626d tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:49.599191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420a8d4-2051-4ec2-88f9-629e0d87626d tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:49.604139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d0866-cdd0-43fa-940e-73ed7a007c01 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:49.604537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420a8d4-2051-4ec2-88f9-629e0d87626d tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:44:49.604862 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420a8d4-2051-4ec2-88f9-629e0d87626d tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:49.606651 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420a8d4-2051-4ec2-88f9-629e0d87626d tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:49.630820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 944/1886] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:44:49.633522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420a8d4-2051-4ec2-88f9-629e0d87626d tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:49.636986 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d0866-cdd0-43fa-940e-73ed7a007c01 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:49.642569 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da497684-5cbd-46d4-ad96-1db516aa8680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:49.647630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da497684-5cbd-46d4-ad96-1db516aa8680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:49.647630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da497684-5cbd-46d4-ad96-1db516aa8680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:49.647630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da497684-5cbd-46d4-ad96-1db516aa8680 admin admin] PATH_INFO: `/v3/projects/2404ff01f2f9416faa9910606af7bd87/users/853d107681d641ea95179e81d834faab/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:49.650106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da497684-5cbd-46d4-ad96-1db516aa8680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2404ff01f2f9416faa9910606af7bd87, user_id=853d107681d641ea95179e81d834faab, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:49.656011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da497684-5cbd-46d4-ad96-1db516aa8680 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:49.702502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420a8d4-2051-4ec2-88f9-629e0d87626d tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:49.718443 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 943/1887] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:49 2022] PUT /identity/v3/projects/2404ff01f2f9416faa9910606af7bd87/users/853d107681d641ea95179e81d834faab/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:49.735873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:49.740766 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f5be4-54e8-4693-83f3-8c48f034cbdf None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:49.741451 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f5be4-54e8-4693-83f3-8c48f034cbdf None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:49.742082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f5be4-54e8-4693-83f3-8c48f034cbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:49.821804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-174f5be4-54e8-4693-83f3-8c48f034cbdf None None] MFA Rules not processed for user `853d107681d641ea95179e81d834faab`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:49.825176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-174f5be4-54e8-4693-83f3-8c48f034cbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:49.860079 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a51473-4ecf-44d3-8a99-98fcc231d485 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b63970885549f1832fd7ff51d52ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f60008a12be54711b174a928be3dfc03', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:49.860845 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a51473-4ecf-44d3-8a99-98fcc231d485 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:49.861543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a51473-4ecf-44d3-8a99-98fcc231d485 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:49.861656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a51473-4ecf-44d3-8a99-98fcc231d485 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:49.864045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a51473-4ecf-44d3-8a99-98fcc231d485 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:49.871730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a51473-4ecf-44d3-8a99-98fcc231d485 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:49.912343 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 945/1888] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:49 2022] POST /identity/v3/projects => generated 355 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:44:49.918122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a51473-4ecf-44d3-8a99-98fcc231d485 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:49.967229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 944/1889] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:49.990096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f5be4-54e8-4693-83f3-8c48f034cbdf None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:49.992585 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-174f5be4-54e8-4693-83f3-8c48f034cbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:50.010866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:50.083198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a51473-4ecf-44d3-8a99-98fcc231d485 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:50.091191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f453d3-53b0-4e84-8cba-2a4b60afc848 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b63970885549f1832fd7ff51d52ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f60008a12be54711b174a928be3dfc03', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:50.092195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f453d3-53b0-4e84-8cba-2a4b60afc848 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:50.092674 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f453d3-53b0-4e84-8cba-2a4b60afc848 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:50.093131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f453d3-53b0-4e84-8cba-2a4b60afc848 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:50.093552 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f5be4-54e8-4693-83f3-8c48f034cbdf None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:50.094415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f453d3-53b0-4e84-8cba-2a4b60afc848 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:44:50.094883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f453d3-53b0-4e84-8cba-2a4b60afc848 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:50.098427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f453d3-53b0-4e84-8cba-2a4b60afc848 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:50.121964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 946/1890] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:44:50.182481 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b7367-0497-4354-97d3-dd0eec53f569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:50.182826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b7367-0497-4354-97d3-dd0eec53f569 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:50.184519 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b7367-0497-4354-97d3-dd0eec53f569 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:50.184519 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b7367-0497-4354-97d3-dd0eec53f569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:50.188150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f453d3-53b0-4e84-8cba-2a4b60afc848 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:50.203527 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f453d3-53b0-4e84-8cba-2a4b60afc848 tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:50.212813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d574a88-fdc5-4593-9709-7fbdec82765c tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b63970885549f1832fd7ff51d52ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f60008a12be54711b174a928be3dfc03', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:50.213636 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d574a88-fdc5-4593-9709-7fbdec82765c tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:50.213960 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d574a88-fdc5-4593-9709-7fbdec82765c tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:50.214313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d574a88-fdc5-4593-9709-7fbdec82765c tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] PATH_INFO: `/v3/projects/f9c1efa8957a4aceb2677c64809d27e4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:50.215732 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d574a88-fdc5-4593-9709-7fbdec82765c tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f9c1efa8957a4aceb2677c64809d27e4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:50.218245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d574a88-fdc5-4593-9709-7fbdec82765c tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:50.297624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b7367-0497-4354-97d3-dd0eec53f569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:50.300916 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b7367-0497-4354-97d3-dd0eec53f569 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:50.306701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2d574a88-fdc5-4593-9709-7fbdec82765c tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c1efa8957a4aceb2677c64809d27e4', 'request_id': 'req-2d574a88-fdc5-4593-9709-7fbdec82765c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:50.317307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2d574a88-fdc5-4593-9709-7fbdec82765c tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c1efa8957a4aceb2677c64809d27e4', 'request_id': 'req-2d574a88-fdc5-4593-9709-7fbdec82765c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:50.326129 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2d574a88-fdc5-4593-9709-7fbdec82765c tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c1efa8957a4aceb2677c64809d27e4', 'request_id': 'req-2d574a88-fdc5-4593-9709-7fbdec82765c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:50.334759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 945/1891] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:50.339535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2d574a88-fdc5-4593-9709-7fbdec82765c tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] The token cache is being invalidate because project f9c1efa8957a4aceb2677c64809d27e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:50.339870 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2d574a88-fdc5-4593-9709-7fbdec82765c tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:50.340574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2d574a88-fdc5-4593-9709-7fbdec82765c tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:50.346379 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 947/1892] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:50 2022] DELETE /identity/v3/projects/f9c1efa8957a4aceb2677c64809d27e4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:50.474620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b7367-0497-4354-97d3-dd0eec53f569 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:50.493051 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b7367-0497-4354-97d3-dd0eec53f569 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:50.660443 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400cd8f-3949-481c-82f6-314ee9a499e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:50.661657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400cd8f-3949-481c-82f6-314ee9a499e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:50.662186 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400cd8f-3949-481c-82f6-314ee9a499e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:50.663025 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400cd8f-3949-481c-82f6-314ee9a499e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:50.666558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7400cd8f-3949-481c-82f6-314ee9a499e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:50.742008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:50.750567 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d574a88-fdc5-4593-9709-7fbdec82765c tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:50.753220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2d574a88-fdc5-4593-9709-7fbdec82765c tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:50.784268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:50.822359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400cd8f-3949-481c-82f6-314ee9a499e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:50.825459 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400cd8f-3949-481c-82f6-314ee9a499e1 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:50.844797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 946/1893] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:50.873182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d574a88-fdc5-4593-9709-7fbdec82765c tempest-VolumeQuotaClassesTest-846053674 tempest-VolumeQuotaClassesTest-846053674-project-admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:50.880850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a264a65-ffff-43d9-a84c-20c90e303814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:50.882199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a264a65-ffff-43d9-a84c-20c90e303814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:50.882663 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a264a65-ffff-43d9-a84c-20c90e303814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:50.883029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a264a65-ffff-43d9-a84c-20c90e303814 admin admin] PATH_INFO: `/v3/users/96a10bbafc0d4b51bd2a76c245a2849e` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:50.884415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a264a65-ffff-43d9-a84c-20c90e303814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96a10bbafc0d4b51bd2a76c245a2849e)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:50.886901 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a264a65-ffff-43d9-a84c-20c90e303814 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:50.973972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8a264a65-ffff-43d9-a84c-20c90e303814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96a10bbafc0d4b51bd2a76c245a2849e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:51.009499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8a264a65-ffff-43d9-a84c-20c90e303814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96a10bbafc0d4b51bd2a76c245a2849e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:51.010577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8a264a65-ffff-43d9-a84c-20c90e303814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96a10bbafc0d4b51bd2a76c245a2849e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:51.023076 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8a264a65-ffff-43d9-a84c-20c90e303814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96a10bbafc0d4b51bd2a76c245a2849e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:51.041379 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8a264a65-ffff-43d9-a84c-20c90e303814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96a10bbafc0d4b51bd2a76c245a2849e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:51.058069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 948/1894] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:50 2022] DELETE /identity/v3/users/96a10bbafc0d4b51bd2a76c245a2849e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:51.063033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5169d5-b680-40aa-82ea-2c105bfbe529 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:51.063391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5169d5-b680-40aa-82ea-2c105bfbe529 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:51.063704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5169d5-b680-40aa-82ea-2c105bfbe529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:51.096682 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5f5169d5-b680-40aa-82ea-2c105bfbe529 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:51.099221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5f5169d5-b680-40aa-82ea-2c105bfbe529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:51.183614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 947/1895] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:51.207946 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a264a65-ffff-43d9-a84c-20c90e303814 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:51.210360 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8a264a65-ffff-43d9-a84c-20c90e303814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:51.320592 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a264a65-ffff-43d9-a84c-20c90e303814 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:51.340312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c95c8-4614-47ac-b941-a7b478752966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:51.341328 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c95c8-4614-47ac-b941-a7b478752966 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:51.341996 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c95c8-4614-47ac-b941-a7b478752966 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:51.342676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c95c8-4614-47ac-b941-a7b478752966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:51.361829 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c95c8-4614-47ac-b941-a7b478752966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:51.365251 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c95c8-4614-47ac-b941-a7b478752966 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:51.395620 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 949/1896] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:51.756463 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5169d5-b680-40aa-82ea-2c105bfbe529 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:51.758107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5f5169d5-b680-40aa-82ea-2c105bfbe529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:51.778821 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5169d5-b680-40aa-82ea-2c105bfbe529 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:51.788680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305d7a5-70a9-4f43-81a8-c94b42ab7210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:51.790637 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305d7a5-70a9-4f43-81a8-c94b42ab7210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:51.791201 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305d7a5-70a9-4f43-81a8-c94b42ab7210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:51.791754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305d7a5-70a9-4f43-81a8-c94b42ab7210 admin admin] PATH_INFO: `/v3/projects/50faea3a8cee4d1089b561b2f2d8bdce` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:51.794972 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305d7a5-70a9-4f43-81a8-c94b42ab7210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50faea3a8cee4d1089b561b2f2d8bdce)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:51.798619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305d7a5-70a9-4f43-81a8-c94b42ab7210 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:51.873764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d305d7a5-70a9-4f43-81a8-c94b42ab7210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50faea3a8cee4d1089b561b2f2d8bdce', 'request_id': 'req-d305d7a5-70a9-4f43-81a8-c94b42ab7210'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:51.886097 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d305d7a5-70a9-4f43-81a8-c94b42ab7210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50faea3a8cee4d1089b561b2f2d8bdce', 'request_id': 'req-d305d7a5-70a9-4f43-81a8-c94b42ab7210'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:51.899895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d305d7a5-70a9-4f43-81a8-c94b42ab7210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50faea3a8cee4d1089b561b2f2d8bdce', 'request_id': 'req-d305d7a5-70a9-4f43-81a8-c94b42ab7210'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:51.915348 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d305d7a5-70a9-4f43-81a8-c94b42ab7210 admin admin] The token cache is being invalidate because project 50faea3a8cee4d1089b561b2f2d8bdce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:51.916322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d305d7a5-70a9-4f43-81a8-c94b42ab7210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:51.917056 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d305d7a5-70a9-4f43-81a8-c94b42ab7210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:51.918702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 948/1897] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:51 2022] DELETE /identity/v3/projects/50faea3a8cee4d1089b561b2f2d8bdce => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:51.923011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c95c8-4614-47ac-b941-a7b478752966 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:51.940625 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c95c8-4614-47ac-b941-a7b478752966 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:51.999404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:52.036053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedeaba-7b32-428b-bcd1-de8a8a69ffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:52.037149 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedeaba-7b32-428b-bcd1-de8a8a69ffc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:52.037702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedeaba-7b32-428b-bcd1-de8a8a69ffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:52.038212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedeaba-7b32-428b-bcd1-de8a8a69ffc8 admin admin] PATH_INFO: `/v3/users/f1b63970885549f1832fd7ff51d52ea4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:52.040511 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedeaba-7b32-428b-bcd1-de8a8a69ffc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b63970885549f1832fd7ff51d52ea4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:52.044662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedeaba-7b32-428b-bcd1-de8a8a69ffc8 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:52.140852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4bedeaba-7b32-428b-bcd1-de8a8a69ffc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b63970885549f1832fd7ff51d52ea4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:52.170356 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4bedeaba-7b32-428b-bcd1-de8a8a69ffc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b63970885549f1832fd7ff51d52ea4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:52.171272 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4bedeaba-7b32-428b-bcd1-de8a8a69ffc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b63970885549f1832fd7ff51d52ea4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:52.190293 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4bedeaba-7b32-428b-bcd1-de8a8a69ffc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b63970885549f1832fd7ff51d52ea4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:52.210595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4bedeaba-7b32-428b-bcd1-de8a8a69ffc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b63970885549f1832fd7ff51d52ea4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:52.228637 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 950/1898] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:51 2022] DELETE /identity/v3/users/f1b63970885549f1832fd7ff51d52ea4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:52.746264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305d7a5-70a9-4f43-81a8-c94b42ab7210 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:52.749023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d305d7a5-70a9-4f43-81a8-c94b42ab7210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:52.779390 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305d7a5-70a9-4f43-81a8-c94b42ab7210 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:52.869314 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:52.904299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9342a-c725-414f-8363-4b5896e9e62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:52.905598 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9342a-c725-414f-8363-4b5896e9e62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:52.906045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9342a-c725-414f-8363-4b5896e9e62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:52.906510 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9342a-c725-414f-8363-4b5896e9e62c admin admin] PATH_INFO: `/v3/projects/f60008a12be54711b174a928be3dfc03` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:52.909111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9342a-c725-414f-8363-4b5896e9e62c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f60008a12be54711b174a928be3dfc03)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:52.913807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9342a-c725-414f-8363-4b5896e9e62c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:53.048123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-29b9342a-c725-414f-8363-4b5896e9e62c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f60008a12be54711b174a928be3dfc03', 'request_id': 'req-29b9342a-c725-414f-8363-4b5896e9e62c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:53.063273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-29b9342a-c725-414f-8363-4b5896e9e62c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f60008a12be54711b174a928be3dfc03', 'request_id': 'req-29b9342a-c725-414f-8363-4b5896e9e62c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:53.079693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-29b9342a-c725-414f-8363-4b5896e9e62c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f60008a12be54711b174a928be3dfc03', 'request_id': 'req-29b9342a-c725-414f-8363-4b5896e9e62c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:53.096617 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-29b9342a-c725-414f-8363-4b5896e9e62c admin admin] The token cache is being invalidate because project f60008a12be54711b174a928be3dfc03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:53.097047 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-29b9342a-c725-414f-8363-4b5896e9e62c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:53.098525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-29b9342a-c725-414f-8363-4b5896e9e62c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:53.100669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 949/1899] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:52 2022] DELETE /identity/v3/projects/f60008a12be54711b174a928be3dfc03 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:53.108017 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0799cb-ee06-4460-b805-7d332678c29b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:53.108833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0799cb-ee06-4460-b805-7d332678c29b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:53.108984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0799cb-ee06-4460-b805-7d332678c29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:53.133758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5d0799cb-ee06-4460-b805-7d332678c29b None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:53.135103 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5d0799cb-ee06-4460-b805-7d332678c29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:53.226299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 951/1900] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:53.237088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b36afa-27c7-4c9b-a326-0458171ee307 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:53.237357 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b36afa-27c7-4c9b-a326-0458171ee307 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:53.237685 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b36afa-27c7-4c9b-a326-0458171ee307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:53.289034 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-20b36afa-27c7-4c9b-a326-0458171ee307 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:53.291713 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-20b36afa-27c7-4c9b-a326-0458171ee307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:53.317551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 950/1901] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:53.321794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0799cb-ee06-4460-b805-7d332678c29b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:53.334285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0799cb-ee06-4460-b805-7d332678c29b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:53.344911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676f69d-0a63-4ee2-8093-fba3f51cbb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:53.346032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676f69d-0a63-4ee2-8093-fba3f51cbb99 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:53.346451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676f69d-0a63-4ee2-8093-fba3f51cbb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:53.346945 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676f69d-0a63-4ee2-8093-fba3f51cbb99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:53.348827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676f69d-0a63-4ee2-8093-fba3f51cbb99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:44:53.349890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676f69d-0a63-4ee2-8093-fba3f51cbb99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:53.351848 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676f69d-0a63-4ee2-8093-fba3f51cbb99 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:53.373533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 952/1902] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:44:53.380332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b36afa-27c7-4c9b-a326-0458171ee307 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:53.393391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b36afa-27c7-4c9b-a326-0458171ee307 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:53.404675 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269fde9-a1fb-44d3-96bd-93c3923626c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:53.406327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269fde9-a1fb-44d3-96bd-93c3923626c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:53.407053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269fde9-a1fb-44d3-96bd-93c3923626c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:53.407567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269fde9-a1fb-44d3-96bd-93c3923626c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:53.409082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269fde9-a1fb-44d3-96bd-93c3923626c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:53.415339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269fde9-a1fb-44d3-96bd-93c3923626c5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:53.484861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 951/1903] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:53 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:53.489684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676f69d-0a63-4ee2-8093-fba3f51cbb99 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:53.500607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676f69d-0a63-4ee2-8093-fba3f51cbb99 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:53.601681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:53.641194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53124e0e-bddc-4cf2-883b-c29c671d4121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:53.642188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53124e0e-bddc-4cf2-883b-c29c671d4121 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:53.642792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53124e0e-bddc-4cf2-883b-c29c671d4121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:53.643229 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53124e0e-bddc-4cf2-883b-c29c671d4121 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:53.644637 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53124e0e-bddc-4cf2-883b-c29c671d4121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:53.647995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53124e0e-bddc-4cf2-883b-c29c671d4121 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:53.700222 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 953/1904] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:53 2022] POST /identity/v3/users => generated 292 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:53.702029 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269fde9-a1fb-44d3-96bd-93c3923626c5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:53.718504 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269fde9-a1fb-44d3-96bd-93c3923626c5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:53.727605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d2d94-5179-4712-91a8-a3b4590951a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:53.728260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d2d94-5179-4712-91a8-a3b4590951a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:53.728710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d2d94-5179-4712-91a8-a3b4590951a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:53.728997 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d2d94-5179-4712-91a8-a3b4590951a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:53.741881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d2d94-5179-4712-91a8-a3b4590951a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:53.741881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d2d94-5179-4712-91a8-a3b4590951a5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:53.751502 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 952/1905] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:44:53.752733 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53124e0e-bddc-4cf2-883b-c29c671d4121 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:53.765087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53124e0e-bddc-4cf2-883b-c29c671d4121 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:53.772214 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb127da-b648-4749-b2ec-02d0faa2f17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:53.773365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb127da-b648-4749-b2ec-02d0faa2f17f admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:53.773895 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb127da-b648-4749-b2ec-02d0faa2f17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:53.774325 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb127da-b648-4749-b2ec-02d0faa2f17f admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:53.775254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb127da-b648-4749-b2ec-02d0faa2f17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:53.777289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb127da-b648-4749-b2ec-02d0faa2f17f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:53.796956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 954/1906] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:53.798486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d2d94-5179-4712-91a8-a3b4590951a5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:53.815605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d2d94-5179-4712-91a8-a3b4590951a5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:53.825516 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f5611-bd3f-4883-bc42-8a90960890e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:53.829241 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f5611-bd3f-4883-bc42-8a90960890e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:53.830708 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f5611-bd3f-4883-bc42-8a90960890e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:53.831468 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f5611-bd3f-4883-bc42-8a90960890e8 admin admin] PATH_INFO: `/v3/projects/1740821171454bba927984de60ffd85e/users/b4a50d4261ea437b870110efc239b0e7/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:53.861493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f5611-bd3f-4883-bc42-8a90960890e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1740821171454bba927984de60ffd85e, user_id=b4a50d4261ea437b870110efc239b0e7, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:53.867639 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f5611-bd3f-4883-bc42-8a90960890e8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:53.889275 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 953/1907] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:53 2022] PUT /identity/v3/projects/1740821171454bba927984de60ffd85e/users/b4a50d4261ea437b870110efc239b0e7/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:53.904488 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4d89f-9d5b-459d-89c9-f0968a1b410c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:53.905093 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4d89f-9d5b-459d-89c9-f0968a1b410c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:53.905746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4d89f-9d5b-459d-89c9-f0968a1b410c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:53.960744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-27e4d89f-9d5b-459d-89c9-f0968a1b410c None None] MFA Rules not processed for user `b4a50d4261ea437b870110efc239b0e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:53.962408 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-27e4d89f-9d5b-459d-89c9-f0968a1b410c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:54.095898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 955/1908] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:54.101002 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f5611-bd3f-4883-bc42-8a90960890e8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:54.247093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f5611-bd3f-4883-bc42-8a90960890e8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:54.253261 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4d89f-9d5b-459d-89c9-f0968a1b410c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:54.255853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-27e4d89f-9d5b-459d-89c9-f0968a1b410c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:54.256964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c922e-64f9-4dde-be0f-8ed617f2f4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:54.258356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c922e-64f9-4dde-be0f-8ed617f2f4a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:54.258898 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c922e-64f9-4dde-be0f-8ed617f2f4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:54.259600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c922e-64f9-4dde-be0f-8ed617f2f4a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:54.261061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c922e-64f9-4dde-be0f-8ed617f2f4a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:54.265605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c922e-64f9-4dde-be0f-8ed617f2f4a3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:54.313248 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 954/1909] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:54 2022] POST /identity/v3/projects => generated 381 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:54.313765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4d89f-9d5b-459d-89c9-f0968a1b410c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:54.318137 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c922e-64f9-4dde-be0f-8ed617f2f4a3 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:54.324675 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:54.424511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c922e-64f9-4dde-be0f-8ed617f2f4a3 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:54.432852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da703d3f-c7fa-4d67-b819-5d3ed0500d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:54.434381 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da703d3f-c7fa-4d67-b819-5d3ed0500d2c admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:54.434953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da703d3f-c7fa-4d67-b819-5d3ed0500d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:54.435331 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da703d3f-c7fa-4d67-b819-5d3ed0500d2c admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:54.437129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da703d3f-c7fa-4d67-b819-5d3ed0500d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:54.440327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da703d3f-c7fa-4d67-b819-5d3ed0500d2c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:54.479591 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 955/1910] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:54 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:44:54.481837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da703d3f-c7fa-4d67-b819-5d3ed0500d2c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:54.500058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da703d3f-c7fa-4d67-b819-5d3ed0500d2c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:54.507653 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945dd57-5e4c-4274-bc1b-9b08680ffc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:54.508419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945dd57-5e4c-4274-bc1b-9b08680ffc15 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:54.508705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945dd57-5e4c-4274-bc1b-9b08680ffc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:54.509001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945dd57-5e4c-4274-bc1b-9b08680ffc15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:54.509870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945dd57-5e4c-4274-bc1b-9b08680ffc15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:54.511875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945dd57-5e4c-4274-bc1b-9b08680ffc15 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:54.535155 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 956/1911] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:54.536420 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945dd57-5e4c-4274-bc1b-9b08680ffc15 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:54.567973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945dd57-5e4c-4274-bc1b-9b08680ffc15 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:54.570046 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fd61a-d978-48df-9044-e4faf160735a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:54.570853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fd61a-d978-48df-9044-e4faf160735a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:54.571179 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fd61a-d978-48df-9044-e4faf160735a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:54.571470 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fd61a-d978-48df-9044-e4faf160735a admin admin] PATH_INFO: `/v3/users/8c13a30d8a7c447fb8e4ada755893d78` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:54.572769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7fd61a-d978-48df-9044-e4faf160735a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c13a30d8a7c447fb8e4ada755893d78)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:54.574834 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7fd61a-d978-48df-9044-e4faf160735a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:54.578724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bce163-d3e0-48a5-8d95-ee424effbb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:54.579870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bce163-d3e0-48a5-8d95-ee424effbb18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:54.580160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bce163-d3e0-48a5-8d95-ee424effbb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:54.580603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bce163-d3e0-48a5-8d95-ee424effbb18 admin admin] PATH_INFO: `/v3/projects/c37542251585431d8692e11d97f93236/users/a2067b38098947a5a760a1413ea3c57c/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:54.609174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bce163-d3e0-48a5-8d95-ee424effbb18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c37542251585431d8692e11d97f93236, user_id=a2067b38098947a5a760a1413ea3c57c, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:54.611958 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bce163-d3e0-48a5-8d95-ee424effbb18 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:54.641512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 957/1912] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:54 2022] PUT /identity/v3/projects/c37542251585431d8692e11d97f93236/users/a2067b38098947a5a760a1413ea3c57c/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:44:54.641512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bce163-d3e0-48a5-8d95-ee424effbb18 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:54.712028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5e7fd61a-d978-48df-9044-e4faf160735a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c13a30d8a7c447fb8e4ada755893d78'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:54.722495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:54.738930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5e7fd61a-d978-48df-9044-e4faf160735a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c13a30d8a7c447fb8e4ada755893d78'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:54.740564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5e7fd61a-d978-48df-9044-e4faf160735a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c13a30d8a7c447fb8e4ada755893d78'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:54.755297 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5e7fd61a-d978-48df-9044-e4faf160735a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c13a30d8a7c447fb8e4ada755893d78'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:54.775899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5e7fd61a-d978-48df-9044-e4faf160735a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c13a30d8a7c447fb8e4ada755893d78'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:54.792763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 956/1913] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:54 2022] DELETE /identity/v3/users/8c13a30d8a7c447fb8e4ada755893d78 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:54.793546 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bce163-d3e0-48a5-8d95-ee424effbb18 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:54.799823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aebfa-93f1-4cc0-a720-d546decd834e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:54.800213 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aebfa-93f1-4cc0-a720-d546decd834e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:54.802507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aebfa-93f1-4cc0-a720-d546decd834e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:54.850430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e65aebfa-93f1-4cc0-a720-d546decd834e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:54.853270 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e65aebfa-93f1-4cc0-a720-d546decd834e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:54.916815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3522e3-38aa-47fa-bf9e-9be1cd2fd4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:54.919560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3522e3-38aa-47fa-bf9e-9be1cd2fd4ab admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:54.920229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3522e3-38aa-47fa-bf9e-9be1cd2fd4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:54.922332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3522e3-38aa-47fa-bf9e-9be1cd2fd4ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:54.923789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3522e3-38aa-47fa-bf9e-9be1cd2fd4ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:54.927716 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3522e3-38aa-47fa-bf9e-9be1cd2fd4ab admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:54.942651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:54.965845 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 958/1914] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:54.967397 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3522e3-38aa-47fa-bf9e-9be1cd2fd4ab admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:54.977620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cc3522e3-38aa-47fa-bf9e-9be1cd2fd4ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:55.034046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3522e3-38aa-47fa-bf9e-9be1cd2fd4ab admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:55.051781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee71221-a345-4396-bde2-1e57e5f44d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:55.054293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee71221-a345-4396-bde2-1e57e5f44d0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:55.055297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee71221-a345-4396-bde2-1e57e5f44d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:55.055962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee71221-a345-4396-bde2-1e57e5f44d0f admin admin] PATH_INFO: `/v3/projects/c37542251585431d8692e11d97f93236/users/a2067b38098947a5a760a1413ea3c57c/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:55.061857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee71221-a345-4396-bde2-1e57e5f44d0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c37542251585431d8692e11d97f93236, user_id=a2067b38098947a5a760a1413ea3c57c, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:55.068838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee71221-a345-4396-bde2-1e57e5f44d0f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:55.093641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 957/1915] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:55.115916 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 959/1916] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:54 2022] PUT /identity/v3/projects/c37542251585431d8692e11d97f93236/users/a2067b38098947a5a760a1413ea3c57c/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:55.116173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65aebfa-93f1-4cc0-a720-d546decd834e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:55.116173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e65aebfa-93f1-4cc0-a720-d546decd834e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:55.128018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e0651-4a2e-47d9-81e4-25cc5e73398b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:55.129753 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e0651-4a2e-47d9-81e4-25cc5e73398b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:55.130283 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e0651-4a2e-47d9-81e4-25cc5e73398b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:55.209263 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e65e0651-4a2e-47d9-81e4-25cc5e73398b None None] MFA Rules not processed for user `a2067b38098947a5a760a1413ea3c57c`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:55.210888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e65e0651-4a2e-47d9-81e4-25cc5e73398b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:55.300283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65aebfa-93f1-4cc0-a720-d546decd834e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:55.391582 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 960/1917] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:55.404893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a191d-d539-4138-8b57-95ccb66fa45d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:55.406293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e0651-4a2e-47d9-81e4-25cc5e73398b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:55.406593 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a191d-d539-4138-8b57-95ccb66fa45d service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:55.407830 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a191d-d539-4138-8b57-95ccb66fa45d service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:55.408682 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e65e0651-4a2e-47d9-81e4-25cc5e73398b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:55.409030 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a191d-d539-4138-8b57-95ccb66fa45d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:55.514460 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e0651-4a2e-47d9-81e4-25cc5e73398b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:55.517334 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:55.518572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def6b1f-f9d1-4096-ac1f-403be3d00eec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:55.520407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def6b1f-f9d1-4096-ac1f-403be3d00eec service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:55.520637 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def6b1f-f9d1-4096-ac1f-403be3d00eec service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:55.520914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def6b1f-f9d1-4096-ac1f-403be3d00eec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:55.561496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a191d-d539-4138-8b57-95ccb66fa45d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:55.562424 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def6b1f-f9d1-4096-ac1f-403be3d00eec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:55.563972 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def6b1f-f9d1-4096-ac1f-403be3d00eec service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:55.564151 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a191d-d539-4138-8b57-95ccb66fa45d service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:55.575560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 961/1918] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:44:55.596070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 958/1919] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:55.755394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def6b1f-f9d1-4096-ac1f-403be3d00eec service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:55.759214 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3def6b1f-f9d1-4096-ac1f-403be3d00eec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:55.785651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def6b1f-f9d1-4096-ac1f-403be3d00eec service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:55.793597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9045c-6f70-490b-9cfc-433e21aed2c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:55.794641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9045c-6f70-490b-9cfc-433e21aed2c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:55.795196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9045c-6f70-490b-9cfc-433e21aed2c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:55.795637 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9045c-6f70-490b-9cfc-433e21aed2c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:55.799422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6ec9045c-6f70-490b-9cfc-433e21aed2c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:55.887211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9045c-6f70-490b-9cfc-433e21aed2c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:55.888539 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9045c-6f70-490b-9cfc-433e21aed2c4 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:55.909605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 962/1920] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:44:56.107796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a191d-d539-4138-8b57-95ccb66fa45d service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:56.109504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-739a191d-d539-4138-8b57-95ccb66fa45d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:56.127647 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a191d-d539-4138-8b57-95ccb66fa45d service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:56.142138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:56.143061 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:56.143477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:56.143770 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6 admin admin] PATH_INFO: `/v3/projects/9b28c0f9b2994d8d8f5efeb474e13777` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:56.146154 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b28c0f9b2994d8d8f5efeb474e13777)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:56.149887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:56.255369 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b28c0f9b2994d8d8f5efeb474e13777', 'request_id': 'req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:56.273037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b28c0f9b2994d8d8f5efeb474e13777', 'request_id': 'req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:56.291904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b28c0f9b2994d8d8f5efeb474e13777', 'request_id': 'req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:56.324438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6 admin admin] The token cache is being invalidate because project 9b28c0f9b2994d8d8f5efeb474e13777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:56.325441 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:56.326326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:56.333602 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 959/1921] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:56 2022] DELETE /identity/v3/projects/9b28c0f9b2994d8d8f5efeb474e13777 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:56.336155 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9045c-6f70-490b-9cfc-433e21aed2c4 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:56.365351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9045c-6f70-490b-9cfc-433e21aed2c4 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:56.404418 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:56.455403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:56.462441 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:56.552458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82be95-e5ea-4bea-9216-b227af010975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:56.553149 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82be95-e5ea-4bea-9216-b227af010975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:56.553426 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82be95-e5ea-4bea-9216-b227af010975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:56.553734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82be95-e5ea-4bea-9216-b227af010975 admin admin] PATH_INFO: `/v3/users/853d107681d641ea95179e81d834faab` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:56.557587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82be95-e5ea-4bea-9216-b227af010975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=853d107681d641ea95179e81d834faab)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:56.563592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82be95-e5ea-4bea-9216-b227af010975 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:56.678212 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-bd82be95-e5ea-4bea-9216-b227af010975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '853d107681d641ea95179e81d834faab'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:56.685524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4f4e0-b87d-4dfa-a065-fdc8150e57a6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:56.694857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-bd82be95-e5ea-4bea-9216-b227af010975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '853d107681d641ea95179e81d834faab'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:56.696357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c913d40-b34f-4b8d-954c-3d2f310e4818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:56.697827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c913d40-b34f-4b8d-954c-3d2f310e4818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:56.698698 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c913d40-b34f-4b8d-954c-3d2f310e4818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:56.699558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c913d40-b34f-4b8d-954c-3d2f310e4818 admin admin] PATH_INFO: `/v3/users/6409cf1abe914d3e8aebbde25fda6582` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:56.702403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c913d40-b34f-4b8d-954c-3d2f310e4818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6409cf1abe914d3e8aebbde25fda6582)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:56.707716 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-bd82be95-e5ea-4bea-9216-b227af010975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '853d107681d641ea95179e81d834faab'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:56.708340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c913d40-b34f-4b8d-954c-3d2f310e4818 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:56.708705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-bd82be95-e5ea-4bea-9216-b227af010975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '853d107681d641ea95179e81d834faab'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:56.709641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-bd82be95-e5ea-4bea-9216-b227af010975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '853d107681d641ea95179e81d834faab'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:56.762811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 963/1922] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:56 2022] DELETE /identity/v3/users/853d107681d641ea95179e81d834faab => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:44:56.813069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0c913d40-b34f-4b8d-954c-3d2f310e4818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6409cf1abe914d3e8aebbde25fda6582'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:56.836897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0c913d40-b34f-4b8d-954c-3d2f310e4818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6409cf1abe914d3e8aebbde25fda6582'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:56.837818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0c913d40-b34f-4b8d-954c-3d2f310e4818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6409cf1abe914d3e8aebbde25fda6582'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:56.840087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:56.856748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0c913d40-b34f-4b8d-954c-3d2f310e4818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6409cf1abe914d3e8aebbde25fda6582'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:56.869526 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0c913d40-b34f-4b8d-954c-3d2f310e4818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6409cf1abe914d3e8aebbde25fda6582'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:56.887823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 960/1923] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:56 2022] DELETE /identity/v3/users/6409cf1abe914d3e8aebbde25fda6582 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:44:56.891794 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d622401-fad1-43b7-bb81-febb55c9ee17 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:56.892412 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d622401-fad1-43b7-bb81-febb55c9ee17 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:56.892698 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d622401-fad1-43b7-bb81-febb55c9ee17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:56.915873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8d622401-fad1-43b7-bb81-febb55c9ee17 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:56.917406 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8d622401-fad1-43b7-bb81-febb55c9ee17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:57.027104 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 964/1924] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:57.041511 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c913d40-b34f-4b8d-954c-3d2f310e4818 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:57.043789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0c913d40-b34f-4b8d-954c-3d2f310e4818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:57.252930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c913d40-b34f-4b8d-954c-3d2f310e4818 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:57.257431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66ca96-a19a-4a31-be27-e425cdbce0fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:57.258401 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66ca96-a19a-4a31-be27-e425cdbce0fd service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:57.258851 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66ca96-a19a-4a31-be27-e425cdbce0fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:57.259745 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66ca96-a19a-4a31-be27-e425cdbce0fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:57.276022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66ca96-a19a-4a31-be27-e425cdbce0fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:57.277594 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66ca96-a19a-4a31-be27-e425cdbce0fd service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:57.295955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 961/1925] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:44:57.435847 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d622401-fad1-43b7-bb81-febb55c9ee17 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:57.438696 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8d622401-fad1-43b7-bb81-febb55c9ee17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:57.467039 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d622401-fad1-43b7-bb81-febb55c9ee17 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:57.479966 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65e16f-6f14-4d10-a9db-77b7e293aa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:57.481231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65e16f-6f14-4d10-a9db-77b7e293aa2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:57.482036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65e16f-6f14-4d10-a9db-77b7e293aa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:57.482224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65e16f-6f14-4d10-a9db-77b7e293aa2c admin admin] PATH_INFO: `/v3/projects/2404ff01f2f9416faa9910606af7bd87` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:57.486401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65e16f-6f14-4d10-a9db-77b7e293aa2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2404ff01f2f9416faa9910606af7bd87)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:57.490667 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65e16f-6f14-4d10-a9db-77b7e293aa2c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:57.569802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:57.608510 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dd65e16f-6f14-4d10-a9db-77b7e293aa2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2404ff01f2f9416faa9910606af7bd87', 'request_id': 'req-dd65e16f-6f14-4d10-a9db-77b7e293aa2c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:57.622627 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dd65e16f-6f14-4d10-a9db-77b7e293aa2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2404ff01f2f9416faa9910606af7bd87', 'request_id': 'req-dd65e16f-6f14-4d10-a9db-77b7e293aa2c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:57.655594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dd65e16f-6f14-4d10-a9db-77b7e293aa2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2404ff01f2f9416faa9910606af7bd87', 'request_id': 'req-dd65e16f-6f14-4d10-a9db-77b7e293aa2c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:57.698201 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dd65e16f-6f14-4d10-a9db-77b7e293aa2c admin admin] The token cache is being invalidate because project 2404ff01f2f9416faa9910606af7bd87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:57.699153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dd65e16f-6f14-4d10-a9db-77b7e293aa2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:57.701228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dd65e16f-6f14-4d10-a9db-77b7e293aa2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:57.718045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 965/1926] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:57 2022] DELETE /identity/v3/projects/2404ff01f2f9416faa9910606af7bd87 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:57.718263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a60c71-7b59-4c51-8d4c-6260b05dc84b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:57.718263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a60c71-7b59-4c51-8d4c-6260b05dc84b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:57.718263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a60c71-7b59-4c51-8d4c-6260b05dc84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:57.765205 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-88a60c71-7b59-4c51-8d4c-6260b05dc84b None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:57.767681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-88a60c71-7b59-4c51-8d4c-6260b05dc84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:57.956484 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 962/1927] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:57.968901 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215069c-c186-4946-bec2-99ae1fc5435b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:57.970511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215069c-c186-4946-bec2-99ae1fc5435b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:57.971109 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215069c-c186-4946-bec2-99ae1fc5435b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:58.045846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a60c71-7b59-4c51-8d4c-6260b05dc84b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:58.048371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-88a60c71-7b59-4c51-8d4c-6260b05dc84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:58.101588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a60c71-7b59-4c51-8d4c-6260b05dc84b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:58.106037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7215069c-c186-4946-bec2-99ae1fc5435b None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:58.111108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7215069c-c186-4946-bec2-99ae1fc5435b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:58.149646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868a7a5-b5cd-4d4e-bc11-3cf36c70461b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:58.151121 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868a7a5-b5cd-4d4e-bc11-3cf36c70461b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:58.151869 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868a7a5-b5cd-4d4e-bc11-3cf36c70461b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:58.152815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868a7a5-b5cd-4d4e-bc11-3cf36c70461b admin admin] PATH_INFO: `/v3/projects/6db800219dfe4b508646e0157f584a74` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:58.154272 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 966/1928] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:58.155621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868a7a5-b5cd-4d4e-bc11-3cf36c70461b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6db800219dfe4b508646e0157f584a74)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:58.160025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868a7a5-b5cd-4d4e-bc11-3cf36c70461b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:58.161997 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215069c-c186-4946-bec2-99ae1fc5435b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:58.191033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215069c-c186-4946-bec2-99ae1fc5435b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:58.215269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8671fd-ebe3-4676-9c4f-22b038d5a421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:58.216478 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8671fd-ebe3-4676-9c4f-22b038d5a421 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:58.217066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8671fd-ebe3-4676-9c4f-22b038d5a421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:58.217700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8671fd-ebe3-4676-9c4f-22b038d5a421 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:58.223313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8671fd-ebe3-4676-9c4f-22b038d5a421 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:44:58.224050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8671fd-ebe3-4676-9c4f-22b038d5a421 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:58.232903 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8671fd-ebe3-4676-9c4f-22b038d5a421 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:58.294981 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 967/1929] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:44:58.297174 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8671fd-ebe3-4676-9c4f-22b038d5a421 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:58.348299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8671fd-ebe3-4676-9c4f-22b038d5a421 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:58.353508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e868a7a5-b5cd-4d4e-bc11-3cf36c70461b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6db800219dfe4b508646e0157f584a74', 'request_id': 'req-e868a7a5-b5cd-4d4e-bc11-3cf36c70461b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:58.368929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e868a7a5-b5cd-4d4e-bc11-3cf36c70461b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6db800219dfe4b508646e0157f584a74', 'request_id': 'req-e868a7a5-b5cd-4d4e-bc11-3cf36c70461b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:58.381681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e868a7a5-b5cd-4d4e-bc11-3cf36c70461b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6db800219dfe4b508646e0157f584a74', 'request_id': 'req-e868a7a5-b5cd-4d4e-bc11-3cf36c70461b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:58.399380 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e868a7a5-b5cd-4d4e-bc11-3cf36c70461b admin admin] The token cache is being invalidate because project 6db800219dfe4b508646e0157f584a74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:44:58.400596 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e868a7a5-b5cd-4d4e-bc11-3cf36c70461b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:58.402744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e868a7a5-b5cd-4d4e-bc11-3cf36c70461b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:58.405708 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 963/1930] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:44:58 2022] DELETE /identity/v3/projects/6db800219dfe4b508646e0157f584a74 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:58.413629 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5a82e-5b7c-448f-84c2-a9a15b914182 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:58.414210 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5a82e-5b7c-448f-84c2-a9a15b914182 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:58.414806 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5a82e-5b7c-448f-84c2-a9a15b914182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:58.461445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-61f5a82e-5b7c-448f-84c2-a9a15b914182 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:58.464068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-61f5a82e-5b7c-448f-84c2-a9a15b914182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:58.470737 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:58.576325 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea81b22-afef-409e-888c-00b7f18a91b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:58.577302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea81b22-afef-409e-888c-00b7f18a91b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:58.577685 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea81b22-afef-409e-888c-00b7f18a91b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:58.578060 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea81b22-afef-409e-888c-00b7f18a91b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:58.579134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea81b22-afef-409e-888c-00b7f18a91b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:58.581619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea81b22-afef-409e-888c-00b7f18a91b0 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:58.586720 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 964/1931] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:58.594713 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab936e-257f-4291-a9d5-0d60d64f8797 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:58.595118 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab936e-257f-4291-a9d5-0d60d64f8797 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:58.595473 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab936e-257f-4291-a9d5-0d60d64f8797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:58.627046 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-55ab936e-257f-4291-a9d5-0d60d64f8797 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:58.628571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-55ab936e-257f-4291-a9d5-0d60d64f8797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:58.652006 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 965/1932] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:44:58.653166 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab936e-257f-4291-a9d5-0d60d64f8797 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:58.658871 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 968/1933] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:58 2022] POST /identity/v3/projects => generated 377 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:58.660884 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea81b22-afef-409e-888c-00b7f18a91b0 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:58.669552 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab936e-257f-4291-a9d5-0d60d64f8797 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:58.671908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3ea81b22-afef-409e-888c-00b7f18a91b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:58.830518 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7be8da-ebcb-4d25-8ca3-ac0313ab5b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:58.832587 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7be8da-ebcb-4d25-8ca3-ac0313ab5b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:58.833456 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7be8da-ebcb-4d25-8ca3-ac0313ab5b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:58.834231 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7be8da-ebcb-4d25-8ca3-ac0313ab5b13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:58.836373 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7be8da-ebcb-4d25-8ca3-ac0313ab5b13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:44:58.837486 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7be8da-ebcb-4d25-8ca3-ac0313ab5b13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:58.840362 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7be8da-ebcb-4d25-8ca3-ac0313ab5b13 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:58.844006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea81b22-afef-409e-888c-00b7f18a91b0 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:58.864736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e1cf8-8810-4e38-ae7f-a2bce2277941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:58.865732 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e1cf8-8810-4e38-ae7f-a2bce2277941 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:58.866132 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e1cf8-8810-4e38-ae7f-a2bce2277941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:58.866553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e1cf8-8810-4e38-ae7f-a2bce2277941 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:58.867860 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e1cf8-8810-4e38-ae7f-a2bce2277941 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:58.871605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e1cf8-8810-4e38-ae7f-a2bce2277941 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:58.872357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 966/1934] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:44:58.881894 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7be8da-ebcb-4d25-8ca3-ac0313ab5b13 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:58.908621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7be8da-ebcb-4d25-8ca3-ac0313ab5b13 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:58.925151 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe2eeb-3fbe-48f2-81d2-0e7756a5b39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:58.928654 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe2eeb-3fbe-48f2-81d2-0e7756a5b39f admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:58.930049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe2eeb-3fbe-48f2-81d2-0e7756a5b39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:58.931288 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe2eeb-3fbe-48f2-81d2-0e7756a5b39f admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:58.932690 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe2eeb-3fbe-48f2-81d2-0e7756a5b39f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:58.938074 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe2eeb-3fbe-48f2-81d2-0e7756a5b39f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:58.948615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 969/1935] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:58 2022] POST /identity/v3/users => generated 289 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:58.950523 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e1cf8-8810-4e38-ae7f-a2bce2277941 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:58.987043 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e1cf8-8810-4e38-ae7f-a2bce2277941 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:59.008275 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 967/1936] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:58 2022] POST /identity/v3/projects => generated 361 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:59.016429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe2eeb-3fbe-48f2-81d2-0e7756a5b39f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:59.017497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebffff-4aa5-4ba6-b6c2-d68251149d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:59.018857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebffff-4aa5-4ba6-b6c2-d68251149d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:59.019466 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebffff-4aa5-4ba6-b6c2-d68251149d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:59.019967 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebffff-4aa5-4ba6-b6c2-d68251149d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:59.021316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebffff-4aa5-4ba6-b6c2-d68251149d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:59.039141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebffff-4aa5-4ba6-b6c2-d68251149d7a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:59.057395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:59.073997 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 970/1937] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:59.076020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebffff-4aa5-4ba6-b6c2-d68251149d7a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:59.105654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebffff-4aa5-4ba6-b6c2-d68251149d7a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:59.222917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe2eeb-3fbe-48f2-81d2-0e7756a5b39f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:59.227247 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57de59d-7258-4692-88f7-38434ccc21cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:59.227928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57de59d-7258-4692-88f7-38434ccc21cf admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:59.228239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57de59d-7258-4692-88f7-38434ccc21cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:59.228515 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57de59d-7258-4692-88f7-38434ccc21cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:59.229442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57de59d-7258-4692-88f7-38434ccc21cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:59.232155 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57de59d-7258-4692-88f7-38434ccc21cf admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:59.240020 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80917a5e-394c-48d7-9059-244e9eb6e564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:59.241932 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80917a5e-394c-48d7-9059-244e9eb6e564 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:59.243071 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80917a5e-394c-48d7-9059-244e9eb6e564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:59.244009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80917a5e-394c-48d7-9059-244e9eb6e564 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:59.246080 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80917a5e-394c-48d7-9059-244e9eb6e564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:59.250725 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80917a5e-394c-48d7-9059-244e9eb6e564 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:59.266676 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 971/1938] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:59.267819 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57de59d-7258-4692-88f7-38434ccc21cf admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:59.270215 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e57de59d-7258-4692-88f7-38434ccc21cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:59.289992 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57de59d-7258-4692-88f7-38434ccc21cf admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:59.298600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b1c0a-9cf8-49ce-b0d7-d26b44d2bda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:59.299371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b1c0a-9cf8-49ce-b0d7-d26b44d2bda3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:59.299663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b1c0a-9cf8-49ce-b0d7-d26b44d2bda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:59.300222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b1c0a-9cf8-49ce-b0d7-d26b44d2bda3 admin admin] PATH_INFO: `/v3/users/b4a50d4261ea437b870110efc239b0e7` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:59.302423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b1c0a-9cf8-49ce-b0d7-d26b44d2bda3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4a50d4261ea437b870110efc239b0e7)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:59.308645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b1c0a-9cf8-49ce-b0d7-d26b44d2bda3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:59.322878 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 968/1939] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:44:59 2022] POST /identity/v3/users => generated 281 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:59.325979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80917a5e-394c-48d7-9059-244e9eb6e564 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:59.342475 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:59.351953 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80917a5e-394c-48d7-9059-244e9eb6e564 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:59.360056 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90810fc-c0fc-43bb-9ccc-facf4a109005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:59.360958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90810fc-c0fc-43bb-9ccc-facf4a109005 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:59.361263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90810fc-c0fc-43bb-9ccc-facf4a109005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:59.361571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90810fc-c0fc-43bb-9ccc-facf4a109005 admin admin] PATH_INFO: `/v3/projects/661c413d13a945e390de4172f249cdad/users/b05ed0dd293948c2a9834075222a1137/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:59.383681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90810fc-c0fc-43bb-9ccc-facf4a109005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=661c413d13a945e390de4172f249cdad, user_id=b05ed0dd293948c2a9834075222a1137, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:59.387214 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90810fc-c0fc-43bb-9ccc-facf4a109005 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:59.419553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 969/1940] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:59 2022] PUT /identity/v3/projects/661c413d13a945e390de4172f249cdad/users/b05ed0dd293948c2a9834075222a1137/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:59.419553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90810fc-c0fc-43bb-9ccc-facf4a109005 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:59.430571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-674b1c0a-9cf8-49ce-b0d7-d26b44d2bda3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4a50d4261ea437b870110efc239b0e7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:59.447320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-674b1c0a-9cf8-49ce-b0d7-d26b44d2bda3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4a50d4261ea437b870110efc239b0e7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:59.462227 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-674b1c0a-9cf8-49ce-b0d7-d26b44d2bda3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4a50d4261ea437b870110efc239b0e7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:59.463358 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-674b1c0a-9cf8-49ce-b0d7-d26b44d2bda3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4a50d4261ea437b870110efc239b0e7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:59.464300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-674b1c0a-9cf8-49ce-b0d7-d26b44d2bda3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4a50d4261ea437b870110efc239b0e7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:44:59.533799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90810fc-c0fc-43bb-9ccc-facf4a109005 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:59.539538 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8c185-b240-466e-8714-45f4b8aa1498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:59.540264 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8c185-b240-466e-8714-45f4b8aa1498 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:59.540550 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8c185-b240-466e-8714-45f4b8aa1498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:59.540829 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8c185-b240-466e-8714-45f4b8aa1498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:59.541608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8c185-b240-466e-8714-45f4b8aa1498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:59.543456 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8c185-b240-466e-8714-45f4b8aa1498 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:59.547767 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 972/1941] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:44:59 2022] DELETE /identity/v3/users/b4a50d4261ea437b870110efc239b0e7 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:44:59.552252 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199250b2-c0b6-48c0-872a-a28e226588c2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:59.552864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199250b2-c0b6-48c0-872a-a28e226588c2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:59.553485 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199250b2-c0b6-48c0-872a-a28e226588c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:59.570607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 970/1942] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:59.573039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030d5d6-0a42-4454-8ab8-0ded207f9516 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:59.573448 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030d5d6-0a42-4454-8ab8-0ded207f9516 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:59.573863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030d5d6-0a42-4454-8ab8-0ded207f9516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:59.626054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2030d5d6-0a42-4454-8ab8-0ded207f9516 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:59.634073 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2030d5d6-0a42-4454-8ab8-0ded207f9516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:59.636008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-199250b2-c0b6-48c0-872a-a28e226588c2 None None] MFA Rules not processed for user `b05ed0dd293948c2a9834075222a1137`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:44:59.637511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-199250b2-c0b6-48c0-872a-a28e226588c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:59.750050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:59.760950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 973/1943] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:59.762132 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199250b2-c0b6-48c0-872a-a28e226588c2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:59.765622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-199250b2-c0b6-48c0-872a-a28e226588c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:59.781131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 971/1944] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:44:59.782378 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030d5d6-0a42-4454-8ab8-0ded207f9516 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:59.797161 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199250b2-c0b6-48c0-872a-a28e226588c2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:59.797493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2030d5d6-0a42-4454-8ab8-0ded207f9516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:59.804811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2b1ca-aa6f-4ae6-960d-c66c559ea027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:59.805776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2b1ca-aa6f-4ae6-960d-c66c559ea027 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:59.806165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2b1ca-aa6f-4ae6-960d-c66c559ea027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:59.806550 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2b1ca-aa6f-4ae6-960d-c66c559ea027 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:59.807394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2b1ca-aa6f-4ae6-960d-c66c559ea027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:59.809433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2b1ca-aa6f-4ae6-960d-c66c559ea027 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:59.823002 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 974/1945] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:44:59.824000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2b1ca-aa6f-4ae6-960d-c66c559ea027 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:59.831005 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b3c2b1ca-aa6f-4ae6-960d-c66c559ea027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:44:59.836928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030d5d6-0a42-4454-8ab8-0ded207f9516 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:59.846620 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8087cad-b02c-4900-9806-c45c1df889ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:44:59.848040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8087cad-b02c-4900-9806-c45c1df889ef admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:44:59.848440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8087cad-b02c-4900-9806-c45c1df889ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:44:59.848988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8087cad-b02c-4900-9806-c45c1df889ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:44:59.849621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8087cad-b02c-4900-9806-c45c1df889ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:44:59.851956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8087cad-b02c-4900-9806-c45c1df889ef admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:44:59.901517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 972/1946] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:44:59 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:44:59.901517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8087cad-b02c-4900-9806-c45c1df889ef admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:44:59.947251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2b1ca-aa6f-4ae6-960d-c66c559ea027 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:44:59.991189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:44:59.997391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8087cad-b02c-4900-9806-c45c1df889ef admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:00.032349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e232d0-8e08-4de9-84ea-4cf4a553d353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:00.032952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e232d0-8e08-4de9-84ea-4cf4a553d353 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:00.033851 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e232d0-8e08-4de9-84ea-4cf4a553d353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:00.034469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e232d0-8e08-4de9-84ea-4cf4a553d353 admin admin] PATH_INFO: `/v3/projects/59a754cbb9e746d599fa6a3e89afd322/users/01445a1aa83648eca666c29aabcaacef/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:00.052360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6a581-8e01-4f3d-8193-0239e30683d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:00.053709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6a581-8e01-4f3d-8193-0239e30683d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:00.054110 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6a581-8e01-4f3d-8193-0239e30683d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:00.055406 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6a581-8e01-4f3d-8193-0239e30683d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:00.059653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e232d0-8e08-4de9-84ea-4cf4a553d353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a754cbb9e746d599fa6a3e89afd322, user_id=01445a1aa83648eca666c29aabcaacef, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:00.062839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e232d0-8e08-4de9-84ea-4cf4a553d353 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:00.076189 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6a581-8e01-4f3d-8193-0239e30683d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:00.078426 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6a581-8e01-4f3d-8193-0239e30683d9 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:00.100390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 973/1947] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:44:59 2022] PUT /identity/v3/projects/59a754cbb9e746d599fa6a3e89afd322/users/01445a1aa83648eca666c29aabcaacef/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:00.102156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e232d0-8e08-4de9-84ea-4cf4a553d353 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:00.111761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 975/1948] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:44:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:00.123423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6387c-e6a0-4a46-a500-9ec6e3a6c22f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:00.128437 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6387c-e6a0-4a46-a500-9ec6e3a6c22f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:00.130129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6387c-e6a0-4a46-a500-9ec6e3a6c22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:00.231402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d9f6387c-e6a0-4a46-a500-9ec6e3a6c22f None None] MFA Rules not processed for user `01445a1aa83648eca666c29aabcaacef`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:00.236161 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d9f6387c-e6a0-4a46-a500-9ec6e3a6c22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:00.308600 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e232d0-8e08-4de9-84ea-4cf4a553d353 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:00.319096 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db4c03-c4e2-4df7-b888-303c30aab5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:00.320958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db4c03-c4e2-4df7-b888-303c30aab5bb admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:00.321823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db4c03-c4e2-4df7-b888-303c30aab5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:00.322749 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db4c03-c4e2-4df7-b888-303c30aab5bb admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:00.324143 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db4c03-c4e2-4df7-b888-303c30aab5bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:00.328260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db4c03-c4e2-4df7-b888-303c30aab5bb admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:00.395601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 974/1949] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:00 2022] POST /identity/v3/users => generated 295 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:00.402421 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db4c03-c4e2-4df7-b888-303c30aab5bb admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:00.433414 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db4c03-c4e2-4df7-b888-303c30aab5bb admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:00.438052 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 976/1950] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:00.444057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce58d6-4b49-4d60-901e-5a6b3b76be69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:00.445676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce58d6-4b49-4d60-901e-5a6b3b76be69 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:00.446372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce58d6-4b49-4d60-901e-5a6b3b76be69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:00.447577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce58d6-4b49-4d60-901e-5a6b3b76be69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:00.449119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce58d6-4b49-4d60-901e-5a6b3b76be69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:00.453603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce58d6-4b49-4d60-901e-5a6b3b76be69 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:00.475093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6387c-e6a0-4a46-a500-9ec6e3a6c22f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:00.477385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d9f6387c-e6a0-4a46-a500-9ec6e3a6c22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:00.485542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 975/1951] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:00.493991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce58d6-4b49-4d60-901e-5a6b3b76be69 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:00.531824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce58d6-4b49-4d60-901e-5a6b3b76be69 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:00.544840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939ab45-775d-484c-9399-4c096839cdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:00.547055 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939ab45-775d-484c-9399-4c096839cdaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:00.547517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939ab45-775d-484c-9399-4c096839cdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:00.547978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939ab45-775d-484c-9399-4c096839cdaf admin admin] PATH_INFO: `/v3/projects/b4019536f1da486cbadd801f3b939585/users/1c09fb8f708142099a6805e61df0a8fc/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:00.596846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939ab45-775d-484c-9399-4c096839cdaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4019536f1da486cbadd801f3b939585, user_id=1c09fb8f708142099a6805e61df0a8fc, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:00.600293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:00.611120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939ab45-775d-484c-9399-4c096839cdaf admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:00.639764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 976/1952] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:00 2022] PUT /identity/v3/projects/b4019536f1da486cbadd801f3b939585/users/1c09fb8f708142099a6805e61df0a8fc/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:00.640792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939ab45-775d-484c-9399-4c096839cdaf admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:00.644759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6387c-e6a0-4a46-a500-9ec6e3a6c22f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:00.805367 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea940a-b044-4e94-8a46-f7988432262d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:00.806088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea940a-b044-4e94-8a46-f7988432262d service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:00.806370 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea940a-b044-4e94-8a46-f7988432262d service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:00.806695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea940a-b044-4e94-8a46-f7988432262d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:00.855901 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939ab45-775d-484c-9399-4c096839cdaf admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:00.871273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbca70-6453-4347-a63b-e4766ae18d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:00.873249 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbca70-6453-4347-a63b-e4766ae18d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:00.873606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbca70-6453-4347-a63b-e4766ae18d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:00.875017 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbca70-6453-4347-a63b-e4766ae18d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:00.876318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbca70-6453-4347-a63b-e4766ae18d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:00.879784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbca70-6453-4347-a63b-e4766ae18d9d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:00.899549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 977/1953] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:00.901036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbca70-6453-4347-a63b-e4766ae18d9d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:00.903834 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b2cbca70-6453-4347-a63b-e4766ae18d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:00.909219 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea940a-b044-4e94-8a46-f7988432262d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:00.910937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea940a-b044-4e94-8a46-f7988432262d service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:00.920173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbca70-6453-4347-a63b-e4766ae18d9d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:00.926827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3547b-f2a3-42a0-b0a5-20d5add92b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:00.927510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3547b-f2a3-42a0-b0a5-20d5add92b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:00.927853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3547b-f2a3-42a0-b0a5-20d5add92b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:00.928355 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3547b-f2a3-42a0-b0a5-20d5add92b11 admin admin] PATH_INFO: `/v3/projects/1740821171454bba927984de60ffd85e` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:00.930308 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3547b-f2a3-42a0-b0a5-20d5add92b11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1740821171454bba927984de60ffd85e)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:00.932632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3547b-f2a3-42a0-b0a5-20d5add92b11 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:00.936128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 977/1954] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:00.937492 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea940a-b044-4e94-8a46-f7988432262d service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:00.957490 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea940a-b044-4e94-8a46-f7988432262d service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:00.963224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a46b57-aab4-40a6-ae41-9a05fa103e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:00.964208 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a46b57-aab4-40a6-ae41-9a05fa103e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:00.964710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a46b57-aab4-40a6-ae41-9a05fa103e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:00.965213 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a46b57-aab4-40a6-ae41-9a05fa103e61 admin admin] PATH_INFO: `/v3/projects/b4019536f1da486cbadd801f3b939585/users/1c09fb8f708142099a6805e61df0a8fc/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:00.969430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a46b57-aab4-40a6-ae41-9a05fa103e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4019536f1da486cbadd801f3b939585, user_id=1c09fb8f708142099a6805e61df0a8fc, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:00.975280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a46b57-aab4-40a6-ae41-9a05fa103e61 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:00.981735 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:00.996475 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 978/1955] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:00 2022] PUT /identity/v3/projects/b4019536f1da486cbadd801f3b939585/users/1c09fb8f708142099a6805e61df0a8fc/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:00.998458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a46b57-aab4-40a6-ae41-9a05fa103e61 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:01.029243 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-00e3547b-f2a3-42a0-b0a5-20d5add92b11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1740821171454bba927984de60ffd85e', 'request_id': 'req-00e3547b-f2a3-42a0-b0a5-20d5add92b11'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:01.042731 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-00e3547b-f2a3-42a0-b0a5-20d5add92b11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1740821171454bba927984de60ffd85e', 'request_id': 'req-00e3547b-f2a3-42a0-b0a5-20d5add92b11'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:01.055716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-00e3547b-f2a3-42a0-b0a5-20d5add92b11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1740821171454bba927984de60ffd85e', 'request_id': 'req-00e3547b-f2a3-42a0-b0a5-20d5add92b11'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:01.070519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-00e3547b-f2a3-42a0-b0a5-20d5add92b11 admin admin] The token cache is being invalidate because project 1740821171454bba927984de60ffd85e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:45:01.070941 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-00e3547b-f2a3-42a0-b0a5-20d5add92b11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:01.071769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-00e3547b-f2a3-42a0-b0a5-20d5add92b11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:01.073609 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 978/1956] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:45:00 2022] DELETE /identity/v3/projects/1740821171454bba927984de60ffd85e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:01.075125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1b557-2a74-44c5-a8db-eb6698547c6e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:01.075515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1b557-2a74-44c5-a8db-eb6698547c6e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:01.075906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1b557-2a74-44c5-a8db-eb6698547c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:01.098635 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a46b57-aab4-40a6-ae41-9a05fa103e61 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:01.120182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3da1b557-2a74-44c5-a8db-eb6698547c6e None None] MFA Rules not processed for user `1c09fb8f708142099a6805e61df0a8fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:01.121384 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3da1b557-2a74-44c5-a8db-eb6698547c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:01.158278 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:01.191714 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d5ede-719b-486f-a487-0d653c4703e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:01.192403 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d5ede-719b-486f-a487-0d653c4703e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:01.192680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d5ede-719b-486f-a487-0d653c4703e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:01.192957 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d5ede-719b-486f-a487-0d653c4703e1 admin admin] PATH_INFO: `/v3/users/01445a1aa83648eca666c29aabcaacef` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:01.195169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d5ede-719b-486f-a487-0d653c4703e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01445a1aa83648eca666c29aabcaacef)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:01.197133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d5ede-719b-486f-a487-0d653c4703e1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:01.236341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 979/1957] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:01.238646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1b557-2a74-44c5-a8db-eb6698547c6e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:01.240660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3da1b557-2a74-44c5-a8db-eb6698547c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:01.278275 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1b557-2a74-44c5-a8db-eb6698547c6e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:01.284377 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f45d5ede-719b-486f-a487-0d653c4703e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01445a1aa83648eca666c29aabcaacef'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:01.287674 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11905c4-764f-455b-8c27-09c9344475b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:01.288795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11905c4-764f-455b-8c27-09c9344475b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:01.289296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11905c4-764f-455b-8c27-09c9344475b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:01.289827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11905c4-764f-455b-8c27-09c9344475b0 admin admin] PATH_INFO: `/v3/users/a2067b38098947a5a760a1413ea3c57c` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:01.292755 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11905c4-764f-455b-8c27-09c9344475b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2067b38098947a5a760a1413ea3c57c)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:01.296775 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11905c4-764f-455b-8c27-09c9344475b0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:01.303049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f45d5ede-719b-486f-a487-0d653c4703e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01445a1aa83648eca666c29aabcaacef'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:01.328894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f45d5ede-719b-486f-a487-0d653c4703e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01445a1aa83648eca666c29aabcaacef'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:01.330029 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f45d5ede-719b-486f-a487-0d653c4703e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01445a1aa83648eca666c29aabcaacef'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:01.330885 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f45d5ede-719b-486f-a487-0d653c4703e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01445a1aa83648eca666c29aabcaacef'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:01.364075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 979/1958] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:45:00 2022] DELETE /identity/v3/users/01445a1aa83648eca666c29aabcaacef => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:01.366267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d5ede-719b-486f-a487-0d653c4703e1 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:01.367057 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f45d5ede-719b-486f-a487-0d653c4703e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:01.406122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e11905c4-764f-455b-8c27-09c9344475b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2067b38098947a5a760a1413ea3c57c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:01.430645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e11905c4-764f-455b-8c27-09c9344475b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2067b38098947a5a760a1413ea3c57c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:01.431470 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e11905c4-764f-455b-8c27-09c9344475b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2067b38098947a5a760a1413ea3c57c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:01.451215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e11905c4-764f-455b-8c27-09c9344475b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2067b38098947a5a760a1413ea3c57c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:01.455190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d5ede-719b-486f-a487-0d653c4703e1 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:01.463895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243e530-5413-48aa-b58c-5fcbba8144f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:01.465492 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243e530-5413-48aa-b58c-5fcbba8144f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:01.465864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243e530-5413-48aa-b58c-5fcbba8144f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:01.466311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243e530-5413-48aa-b58c-5fcbba8144f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:01.468543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e11905c4-764f-455b-8c27-09c9344475b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2067b38098947a5a760a1413ea3c57c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:01.469886 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2243e530-5413-48aa-b58c-5fcbba8144f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:01.488786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 980/1959] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:45:01 2022] DELETE /identity/v3/users/a2067b38098947a5a760a1413ea3c57c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:01.491969 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89deef3-127e-4d4b-84cd-2593b9177a56 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:01.493731 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89deef3-127e-4d4b-84cd-2593b9177a56 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:01.498551 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89deef3-127e-4d4b-84cd-2593b9177a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:01.548752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d89deef3-127e-4d4b-84cd-2593b9177a56 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:01.554318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d89deef3-127e-4d4b-84cd-2593b9177a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:01.563029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:01.597129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243e530-5413-48aa-b58c-5fcbba8144f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:01.599657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243e530-5413-48aa-b58c-5fcbba8144f3 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:01.609824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 980/1960] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:01.718776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 981/1961] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:01.739749 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243e530-5413-48aa-b58c-5fcbba8144f3 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:01.741433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2243e530-5413-48aa-b58c-5fcbba8144f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:01.769720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:01.919716 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243e530-5413-48aa-b58c-5fcbba8144f3 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:02.077985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac609b-8616-443b-85e5-efaeea8f471f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:02.078781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac609b-8616-443b-85e5-efaeea8f471f service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:02.079175 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac609b-8616-443b-85e5-efaeea8f471f service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:02.079741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac609b-8616-443b-85e5-efaeea8f471f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:02.092548 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac609b-8616-443b-85e5-efaeea8f471f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:02.094796 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac609b-8616-443b-85e5-efaeea8f471f service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:02.113258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 981/1962] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:02.459139 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89deef3-127e-4d4b-84cd-2593b9177a56 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:02.460678 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d89deef3-127e-4d4b-84cd-2593b9177a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:02.480392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89deef3-127e-4d4b-84cd-2593b9177a56 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:02.493820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:02.495508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:02.496410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:02.497148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d admin admin] PATH_INFO: `/v3/projects/c37542251585431d8692e11d97f93236` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:02.500286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c37542251585431d8692e11d97f93236)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:02.505937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:02.586520 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c37542251585431d8692e11d97f93236', 'request_id': 'req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:02.597613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c37542251585431d8692e11d97f93236', 'request_id': 'req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:02.608175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c37542251585431d8692e11d97f93236', 'request_id': 'req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:02.623741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d admin admin] The token cache is being invalidate because project c37542251585431d8692e11d97f93236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:45:02.624276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:02.625897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:02.633578 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 982/1963] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:45:02 2022] DELETE /identity/v3/projects/c37542251585431d8692e11d97f93236 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:02.637505 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb653f38-f1af-49cf-9b2b-262388dc06d0 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:02.637882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb653f38-f1af-49cf-9b2b-262388dc06d0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:02.638250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb653f38-f1af-49cf-9b2b-262388dc06d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:02.653205 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:02.654858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:02.690864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-eb653f38-f1af-49cf-9b2b-262388dc06d0 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:02.692056 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-eb653f38-f1af-49cf-9b2b-262388dc06d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:02.778105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10689a86-77a4-4b6f-8ef8-ff1a7af15f6d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:02.783386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a472759-6c82-4b6d-880a-94ff8ca114c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:02.784062 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a472759-6c82-4b6d-880a-94ff8ca114c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:02.784344 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a472759-6c82-4b6d-880a-94ff8ca114c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:02.784649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a472759-6c82-4b6d-880a-94ff8ca114c7 admin admin] PATH_INFO: `/v3/projects/59a754cbb9e746d599fa6a3e89afd322` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:02.785948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a472759-6c82-4b6d-880a-94ff8ca114c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a754cbb9e746d599fa6a3e89afd322)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:02.787919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a472759-6c82-4b6d-880a-94ff8ca114c7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:02.793713 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 982/1964] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:02.798027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6222723-2f26-4c74-9a6f-67ba6b2c3b5d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:02.798385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6222723-2f26-4c74-9a6f-67ba6b2c3b5d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:02.798734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6222723-2f26-4c74-9a6f-67ba6b2c3b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:02.827947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f6222723-2f26-4c74-9a6f-67ba6b2c3b5d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:02.829298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f6222723-2f26-4c74-9a6f-67ba6b2c3b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:02.838522 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 983/1965] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:02.843106 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:02.843309 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6222723-2f26-4c74-9a6f-67ba6b2c3b5d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:02.861258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6222723-2f26-4c74-9a6f-67ba6b2c3b5d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:02.873013 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4a472759-6c82-4b6d-880a-94ff8ca114c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a754cbb9e746d599fa6a3e89afd322', 'request_id': 'req-4a472759-6c82-4b6d-880a-94ff8ca114c7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:02.887037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4a472759-6c82-4b6d-880a-94ff8ca114c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a754cbb9e746d599fa6a3e89afd322', 'request_id': 'req-4a472759-6c82-4b6d-880a-94ff8ca114c7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:02.903847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4a472759-6c82-4b6d-880a-94ff8ca114c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a754cbb9e746d599fa6a3e89afd322', 'request_id': 'req-4a472759-6c82-4b6d-880a-94ff8ca114c7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:02.924516 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4a472759-6c82-4b6d-880a-94ff8ca114c7 admin admin] The token cache is being invalidate because project 59a754cbb9e746d599fa6a3e89afd322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:45:02.925195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4a472759-6c82-4b6d-880a-94ff8ca114c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:02.928493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4a472759-6c82-4b6d-880a-94ff8ca114c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:02.932084 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 983/1966] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:45:02 2022] DELETE /identity/v3/projects/59a754cbb9e746d599fa6a3e89afd322 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:45:02.940035 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba82c8d-64bb-494a-b0b1-e9376f02fb02 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:02.940500 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba82c8d-64bb-494a-b0b1-e9376f02fb02 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:02.941860 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba82c8d-64bb-494a-b0b1-e9376f02fb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:02.972299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6ac0c-975c-433b-8f65-11e206361f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:02.973200 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6ac0c-975c-433b-8f65-11e206361f9e admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:02.973520 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6ac0c-975c-433b-8f65-11e206361f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:02.973822 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6ac0c-975c-433b-8f65-11e206361f9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:02.974685 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6ac0c-975c-433b-8f65-11e206361f9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:45:02.975009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6ac0c-975c-433b-8f65-11e206361f9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:02.976465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9ba82c8d-64bb-494a-b0b1-e9376f02fb02 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:02.976793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6ac0c-975c-433b-8f65-11e206361f9e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:02.978281 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9ba82c8d-64bb-494a-b0b1-e9376f02fb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:02.988857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:02.997385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 984/1967] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:45:02.998554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6ac0c-975c-433b-8f65-11e206361f9e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:03.000607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a0a6ac0c-975c-433b-8f65-11e206361f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:03.004187 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 984/1968] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:03.005750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30af47d-2ed6-452c-8054-73901bfdf44e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.006040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30af47d-2ed6-452c-8054-73901bfdf44e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.006315 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30af47d-2ed6-452c-8054-73901bfdf44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.015791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6ac0c-975c-433b-8f65-11e206361f9e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:03.026117 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae018c-24f4-4b25-ad8a-0d4a8ce1a34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:03.026770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae018c-24f4-4b25-ad8a-0d4a8ce1a34f admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.027061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae018c-24f4-4b25-ad8a-0d4a8ce1a34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.027339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae018c-24f4-4b25-ad8a-0d4a8ce1a34f admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.028115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae018c-24f4-4b25-ad8a-0d4a8ce1a34f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:03.029110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a30af47d-2ed6-452c-8054-73901bfdf44e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:03.030375 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a30af47d-2ed6-452c-8054-73901bfdf44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:03.030494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae018c-24f4-4b25-ad8a-0d4a8ce1a34f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:03.038803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 985/1969] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:03.043692 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30af47d-2ed6-452c-8054-73901bfdf44e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:03.054195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30af47d-2ed6-452c-8054-73901bfdf44e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:03.059333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92850eb-7d88-4c61-bf87-72990eee9def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:03.059989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92850eb-7d88-4c61-bf87-72990eee9def admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.060280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92850eb-7d88-4c61-bf87-72990eee9def admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.060544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92850eb-7d88-4c61-bf87-72990eee9def admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.061252 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92850eb-7d88-4c61-bf87-72990eee9def admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:45:03.064689 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92850eb-7d88-4c61-bf87-72990eee9def admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:03.064827 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 985/1970] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:45:02 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:03.064909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92850eb-7d88-4c61-bf87-72990eee9def admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:03.067699 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae018c-24f4-4b25-ad8a-0d4a8ce1a34f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:03.080815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 986/1971] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:45:03.086464 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92850eb-7d88-4c61-bf87-72990eee9def admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:03.101629 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92850eb-7d88-4c61-bf87-72990eee9def admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:03.156949 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae018c-24f4-4b25-ad8a-0d4a8ce1a34f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:03.169959 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896de30-0522-497b-bc55-dfc7a8f215b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:03.171432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896de30-0522-497b-bc55-dfc7a8f215b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.171725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896de30-0522-497b-bc55-dfc7a8f215b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.172009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896de30-0522-497b-bc55-dfc7a8f215b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.173116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896de30-0522-497b-bc55-dfc7a8f215b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:03.176075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896de30-0522-497b-bc55-dfc7a8f215b5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:03.179298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e2c3b-ae79-461f-95b9-5a5960262300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:03.180456 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e2c3b-ae79-461f-95b9-5a5960262300 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.181098 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e2c3b-ae79-461f-95b9-5a5960262300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.181836 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e2c3b-ae79-461f-95b9-5a5960262300 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.183458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e2c3b-ae79-461f-95b9-5a5960262300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:03.188567 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e2c3b-ae79-461f-95b9-5a5960262300 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:03.219973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 986/1972] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:03 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:03.226506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896de30-0522-497b-bc55-dfc7a8f215b5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:03.248268 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896de30-0522-497b-bc55-dfc7a8f215b5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:03.258274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca03951-ea64-4119-9b9f-2989b1477f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:03.259013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca03951-ea64-4119-9b9f-2989b1477f9e admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.259347 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca03951-ea64-4119-9b9f-2989b1477f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.259651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca03951-ea64-4119-9b9f-2989b1477f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.260564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca03951-ea64-4119-9b9f-2989b1477f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:03.262606 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca03951-ea64-4119-9b9f-2989b1477f9e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:03.267152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 987/1973] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:45:03 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:45:03.268775 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e2c3b-ae79-461f-95b9-5a5960262300 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:03.275153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 987/1974] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:45:03.279118 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca03951-ea64-4119-9b9f-2989b1477f9e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:03.315640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca03951-ea64-4119-9b9f-2989b1477f9e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:03.365296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e2c3b-ae79-461f-95b9-5a5960262300 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:03.373271 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a543d-31ec-401a-80ea-915793e1abed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:03.374034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a543d-31ec-401a-80ea-915793e1abed admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.374374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a543d-31ec-401a-80ea-915793e1abed admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.374713 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a543d-31ec-401a-80ea-915793e1abed admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.375477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a543d-31ec-401a-80ea-915793e1abed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:03.377597 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a543d-31ec-401a-80ea-915793e1abed admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:03.412848 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 988/1975] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:03 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:45:03.417275 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a543d-31ec-401a-80ea-915793e1abed admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:03.437259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a543d-31ec-401a-80ea-915793e1abed admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:03.443453 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81eb6b-9de0-4001-ab54-0eae5775b0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:03.443967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81eb6b-9de0-4001-ab54-0eae5775b0cf admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.444274 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81eb6b-9de0-4001-ab54-0eae5775b0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.444536 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81eb6b-9de0-4001-ab54-0eae5775b0cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.445364 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81eb6b-9de0-4001-ab54-0eae5775b0cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:03.447048 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81eb6b-9de0-4001-ab54-0eae5775b0cf admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:03.453564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:03.462242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 989/1976] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:03.471551 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81eb6b-9de0-4001-ab54-0eae5775b0cf admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:03.473610 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199d380-2405-4a00-9a58-a78e3bb52a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:03.474280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199d380-2405-4a00-9a58-a78e3bb52a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.474567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199d380-2405-4a00-9a58-a78e3bb52a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.474844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199d380-2405-4a00-9a58-a78e3bb52a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.475566 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199d380-2405-4a00-9a58-a78e3bb52a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:03.477302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199d380-2405-4a00-9a58-a78e3bb52a4b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:03.489027 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81eb6b-9de0-4001-ab54-0eae5775b0cf admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:03.499679 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e5a83-ce17-4da8-bc6d-75f36618fbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:03.500482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e5a83-ce17-4da8-bc6d-75f36618fbdd admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.500740 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e5a83-ce17-4da8-bc6d-75f36618fbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.501088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e5a83-ce17-4da8-bc6d-75f36618fbdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.501939 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e5a83-ce17-4da8-bc6d-75f36618fbdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:03.511458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 988/1977] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:03.512200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e5a83-ce17-4da8-bc6d-75f36618fbdd admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:03.517416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199d380-2405-4a00-9a58-a78e3bb52a4b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:03.539299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 990/1978] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:03.540470 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199d380-2405-4a00-9a58-a78e3bb52a4b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:03.543614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e5a83-ce17-4da8-bc6d-75f36618fbdd admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:03.553994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930c16f-ca74-4403-a870-8ca9554fd00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:03.554714 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930c16f-ca74-4403-a870-8ca9554fd00d admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.555617 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930c16f-ca74-4403-a870-8ca9554fd00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.555976 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930c16f-ca74-4403-a870-8ca9554fd00d admin admin] PATH_INFO: `/v3/projects/c517457e664a42118ea0a0cdb1523dd5/users/b7fb2b5c4c934d229033282b5a813d09/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.558506 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e5a83-ce17-4da8-bc6d-75f36618fbdd admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:03.583105 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930c16f-ca74-4403-a870-8ca9554fd00d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c517457e664a42118ea0a0cdb1523dd5, user_id=b7fb2b5c4c934d229033282b5a813d09, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:03.587060 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cccc0e-d172-46c6-91f3-a98e5e89f2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:03.587328 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930c16f-ca74-4403-a870-8ca9554fd00d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:03.588845 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cccc0e-d172-46c6-91f3-a98e5e89f2e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.589490 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cccc0e-d172-46c6-91f3-a98e5e89f2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.590864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cccc0e-d172-46c6-91f3-a98e5e89f2e0 admin admin] PATH_INFO: `/v3/projects/63222ebbec484fe6942b139fd31d0107/users/920ede2286ff47df833e3055860e9c83/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.615243 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 989/1979] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:03 2022] PUT /identity/v3/projects/c517457e664a42118ea0a0cdb1523dd5/users/b7fb2b5c4c934d229033282b5a813d09/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:45:03.616215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cccc0e-d172-46c6-91f3-a98e5e89f2e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63222ebbec484fe6942b139fd31d0107, user_id=920ede2286ff47df833e3055860e9c83, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:03.619370 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cccc0e-d172-46c6-91f3-a98e5e89f2e0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:03.622965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a6efa-5473-454f-b19a-df40b6e3d1a1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.623304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a6efa-5473-454f-b19a-df40b6e3d1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.623583 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a6efa-5473-454f-b19a-df40b6e3d1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.649547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 991/1980] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:03 2022] PUT /identity/v3/projects/63222ebbec484fe6942b139fd31d0107/users/920ede2286ff47df833e3055860e9c83/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:45:03.661190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-727a6efa-5473-454f-b19a-df40b6e3d1a1 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:03.662508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-727a6efa-5473-454f-b19a-df40b6e3d1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:03.669442 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7ae3c-cbf9-4c61-8350-acca9b302078 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.670057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7ae3c-cbf9-4c61-8350-acca9b302078 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.671079 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7ae3c-cbf9-4c61-8350-acca9b302078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.693805 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:03.720404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9be7ae3c-cbf9-4c61-8350-acca9b302078 None None] MFA Rules not processed for user `920ede2286ff47df833e3055860e9c83`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:03.721981 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9be7ae3c-cbf9-4c61-8350-acca9b302078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:03.760510 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 990/1981] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:03.774002 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a6efa-5473-454f-b19a-df40b6e3d1a1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:03.777507 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-727a6efa-5473-454f-b19a-df40b6e3d1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:03.845537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 992/1982] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:03.852476 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7ae3c-cbf9-4c61-8350-acca9b302078 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:03.867670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7ae3c-cbf9-4c61-8350-acca9b302078 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:03.867944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a6efa-5473-454f-b19a-df40b6e3d1a1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:03.872824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cad3a-952b-41e0-bfeb-f575c1d18246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:03.873719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cad3a-952b-41e0-bfeb-f575c1d18246 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.874018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cad3a-952b-41e0-bfeb-f575c1d18246 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.874291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb651b37-9f59-41f4-8529-3148055d5d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:03.874478 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cad3a-952b-41e0-bfeb-f575c1d18246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.875180 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb651b37-9f59-41f4-8529-3148055d5d24 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:03.875568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb651b37-9f59-41f4-8529-3148055d5d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:03.875935 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb651b37-9f59-41f4-8529-3148055d5d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:03.876747 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb651b37-9f59-41f4-8529-3148055d5d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:03.878988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb651b37-9f59-41f4-8529-3148055d5d24 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:03.886246 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cad3a-952b-41e0-bfeb-f575c1d18246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:03.887862 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cad3a-952b-41e0-bfeb-f575c1d18246 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:03.899089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 991/1983] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:03.911752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 993/1984] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:45:03 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:45:03.916357 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cad3a-952b-41e0-bfeb-f575c1d18246 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:03.930955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cad3a-952b-41e0-bfeb-f575c1d18246 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:04.026727 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdaa65-60f7-4962-82c8-253acad0f3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:04.027468 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdaa65-60f7-4962-82c8-253acad0f3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:04.027763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdaa65-60f7-4962-82c8-253acad0f3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:04.028042 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdaa65-60f7-4962-82c8-253acad0f3f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:04.028752 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdaa65-60f7-4962-82c8-253acad0f3f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:04.031039 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdaa65-60f7-4962-82c8-253acad0f3f9 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:04.056509 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 992/1985] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:03 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:45:04.060610 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb651b37-9f59-41f4-8529-3148055d5d24 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:04.072267 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb651b37-9f59-41f4-8529-3148055d5d24 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:04.077631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6cc97e-630f-4cc0-95f1-465208595a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:04.078412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cc97e-630f-4cc0-95f1-465208595a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:04.078697 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cc97e-630f-4cc0-95f1-465208595a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:04.079175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cc97e-630f-4cc0-95f1-465208595a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:04.079984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cc97e-630f-4cc0-95f1-465208595a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:04.082319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cc97e-630f-4cc0-95f1-465208595a3f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:04.096324 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 994/1986] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:45:04.102184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdaa65-60f7-4962-82c8-253acad0f3f9 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:04.113958 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdaa65-60f7-4962-82c8-253acad0f3f9 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:04.121404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbeff78-b89e-4a1e-b1db-4032ed29e246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:04.122531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeff78-b89e-4a1e-b1db-4032ed29e246 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:04.122701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeff78-b89e-4a1e-b1db-4032ed29e246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:04.123160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeff78-b89e-4a1e-b1db-4032ed29e246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:04.124271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbeff78-b89e-4a1e-b1db-4032ed29e246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:04.126166 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbeff78-b89e-4a1e-b1db-4032ed29e246 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:04.139829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 993/1987] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:45:04.145027 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6cc97e-630f-4cc0-95f1-465208595a3f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:04.157935 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6cc97e-630f-4cc0-95f1-465208595a3f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:04.164718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97505f51-6803-4f7f-b560-a24e5c155314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:04.165816 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97505f51-6803-4f7f-b560-a24e5c155314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:04.166181 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97505f51-6803-4f7f-b560-a24e5c155314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:04.166552 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97505f51-6803-4f7f-b560-a24e5c155314 admin admin] PATH_INFO: `/v3/projects/0cf4305770f4438baebbc5f658903a8f/users/c43a69201b694568b926c110696039bc/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:04.184745 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97505f51-6803-4f7f-b560-a24e5c155314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf4305770f4438baebbc5f658903a8f, user_id=c43a69201b694568b926c110696039bc, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:04.187743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97505f51-6803-4f7f-b560-a24e5c155314 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:04.208636 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 995/1988] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:04 2022] PUT /identity/v3/projects/0cf4305770f4438baebbc5f658903a8f/users/c43a69201b694568b926c110696039bc/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:04.221446 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03cd14-d606-46f6-81f4-8f3eaaf17a13 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:04.221933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03cd14-d606-46f6-81f4-8f3eaaf17a13 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:04.223100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03cd14-d606-46f6-81f4-8f3eaaf17a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:04.272248 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-cd03cd14-d606-46f6-81f4-8f3eaaf17a13 None None] MFA Rules not processed for user `c43a69201b694568b926c110696039bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:04.274704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cd03cd14-d606-46f6-81f4-8f3eaaf17a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:04.360605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:04.401846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 994/1989] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:04.403500 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97505f51-6803-4f7f-b560-a24e5c155314 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:04.494281 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:04.499352 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97505f51-6803-4f7f-b560-a24e5c155314 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:04.507563 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d660a20-75b4-4a2d-ac4d-9502046a9445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:04.508554 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d660a20-75b4-4a2d-ac4d-9502046a9445 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:04.508904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d660a20-75b4-4a2d-ac4d-9502046a9445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:04.509257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d660a20-75b4-4a2d-ac4d-9502046a9445 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:04.509987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d660a20-75b4-4a2d-ac4d-9502046a9445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:04.511969 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d660a20-75b4-4a2d-ac4d-9502046a9445 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:04.549548 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 996/1990] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:45:04 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:04.554528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03cd14-d606-46f6-81f4-8f3eaaf17a13 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:04.567027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03cd14-d606-46f6-81f4-8f3eaaf17a13 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:04.647242 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80fac5f-0a94-4d18-a4c3-f6ebe35181e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:04.648047 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fac5f-0a94-4d18-a4c3-f6ebe35181e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:04.648435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fac5f-0a94-4d18-a4c3-f6ebe35181e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:04.648933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fac5f-0a94-4d18-a4c3-f6ebe35181e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:04.649795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80fac5f-0a94-4d18-a4c3-f6ebe35181e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:04.651886 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80fac5f-0a94-4d18-a4c3-f6ebe35181e4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:04.689651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 995/1991] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:04 2022] POST /identity/v3/users => generated 299 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:45:04.692730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d660a20-75b4-4a2d-ac4d-9502046a9445 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:04.717307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d660a20-75b4-4a2d-ac4d-9502046a9445 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:04.730707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0aeb33-8996-4ec8-afed-d17a549e9a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:04.731910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0aeb33-8996-4ec8-afed-d17a549e9a89 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:04.732424 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0aeb33-8996-4ec8-afed-d17a549e9a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:04.733203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0aeb33-8996-4ec8-afed-d17a549e9a89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:04.734715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0aeb33-8996-4ec8-afed-d17a549e9a89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:04.738314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0aeb33-8996-4ec8-afed-d17a549e9a89 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:04.772617 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cb063-2dbb-401a-a500-cc6ef7cbc890 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:04.773163 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cb063-2dbb-401a-a500-cc6ef7cbc890 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:04.773705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cb063-2dbb-401a-a500-cc6ef7cbc890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:04.791184 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 997/1992] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:04.792779 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0aeb33-8996-4ec8-afed-d17a549e9a89 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:04.808231 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0aeb33-8996-4ec8-afed-d17a549e9a89 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:04.815111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac07b80-f959-4090-b1d6-640a13a20f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:04.816162 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac07b80-f959-4090-b1d6-640a13a20f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:04.816653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac07b80-f959-4090-b1d6-640a13a20f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:04.818338 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac07b80-f959-4090-b1d6-640a13a20f45 admin admin] PATH_INFO: `/v3/projects/bfed0a601d1d4c5aa7e11fa7df1e613e/users/a4b0ef2f84b84215887c79ae9c5bfcd2/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:04.832850 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3d3cb063-2dbb-401a-a500-cc6ef7cbc890 None None] MFA Rules not processed for user `b7fb2b5c4c934d229033282b5a813d09`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:04.834332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3d3cb063-2dbb-401a-a500-cc6ef7cbc890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:04.839409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac07b80-f959-4090-b1d6-640a13a20f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfed0a601d1d4c5aa7e11fa7df1e613e, user_id=a4b0ef2f84b84215887c79ae9c5bfcd2, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:04.844529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac07b80-f959-4090-b1d6-640a13a20f45 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:04.887566 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 998/1993] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:04 2022] PUT /identity/v3/projects/bfed0a601d1d4c5aa7e11fa7df1e613e/users/a4b0ef2f84b84215887c79ae9c5bfcd2/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:04.892059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac07b80-f959-4090-b1d6-640a13a20f45 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:04.964227 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 996/1994] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:04.971857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3cb063-2dbb-401a-a500-cc6ef7cbc890 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:04.986807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3cb063-2dbb-401a-a500-cc6ef7cbc890 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:05.004447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac07b80-f959-4090-b1d6-640a13a20f45 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:05.012543 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dec3ec-871d-447a-bd60-32a45b1c0056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:05.014260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dec3ec-871d-447a-bd60-32a45b1c0056 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:05.014674 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dec3ec-871d-447a-bd60-32a45b1c0056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:05.015084 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dec3ec-871d-447a-bd60-32a45b1c0056 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:05.016026 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dec3ec-871d-447a-bd60-32a45b1c0056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:05.018176 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dec3ec-871d-447a-bd60-32a45b1c0056 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:05.047986 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 999/1995] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:05.049173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dec3ec-871d-447a-bd60-32a45b1c0056 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:05.065124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dec3ec-871d-447a-bd60-32a45b1c0056 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:05.071783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcd8cf-3cb8-4d4f-b67a-623a4c1d77d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:05.072495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcd8cf-3cb8-4d4f-b67a-623a4c1d77d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:05.072881 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcd8cf-3cb8-4d4f-b67a-623a4c1d77d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:05.073292 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcd8cf-3cb8-4d4f-b67a-623a4c1d77d7 admin admin] PATH_INFO: `/v3/projects/bfed0a601d1d4c5aa7e11fa7df1e613e/users/a4b0ef2f84b84215887c79ae9c5bfcd2/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:05.074581 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2eddb-6c50-4a2a-9aa0-6eeaca0c96f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:05.075261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2eddb-6c50-4a2a-9aa0-6eeaca0c96f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:05.075548 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2eddb-6c50-4a2a-9aa0-6eeaca0c96f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:05.075798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcd8cf-3cb8-4d4f-b67a-623a4c1d77d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfed0a601d1d4c5aa7e11fa7df1e613e, user_id=a4b0ef2f84b84215887c79ae9c5bfcd2, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:05.076013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2eddb-6c50-4a2a-9aa0-6eeaca0c96f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:05.076765 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2eddb-6c50-4a2a-9aa0-6eeaca0c96f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:05.078956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2eddb-6c50-4a2a-9aa0-6eeaca0c96f9 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:05.080014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcd8cf-3cb8-4d4f-b67a-623a4c1d77d7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:05.085394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:05.105650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1000/1996] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:05 2022] PUT /identity/v3/projects/bfed0a601d1d4c5aa7e11fa7df1e613e/users/a4b0ef2f84b84215887c79ae9c5bfcd2/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:05.107358 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794800aa-9bb4-4636-925e-465df4f14d3d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:05.107652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794800aa-9bb4-4636-925e-465df4f14d3d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:05.108215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794800aa-9bb4-4636-925e-465df4f14d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:05.129651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 997/1997] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:45:04 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:05.134397 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2eddb-6c50-4a2a-9aa0-6eeaca0c96f9 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:05.147812 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-794800aa-9bb4-4636-925e-465df4f14d3d None None] MFA Rules not processed for user `a4b0ef2f84b84215887c79ae9c5bfcd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:05.149873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-794800aa-9bb4-4636-925e-465df4f14d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:05.212774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2eddb-6c50-4a2a-9aa0-6eeaca0c96f9 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:05.218528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef198346-472d-4198-bb36-5bb756e3d389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:05.219532 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef198346-472d-4198-bb36-5bb756e3d389 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:05.219818 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef198346-472d-4198-bb36-5bb756e3d389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:05.220102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef198346-472d-4198-bb36-5bb756e3d389 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:05.220854 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef198346-472d-4198-bb36-5bb756e3d389 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:05.223602 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef198346-472d-4198-bb36-5bb756e3d389 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:05.250966 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 998/1998] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:05 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:45:05.255874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef198346-472d-4198-bb36-5bb756e3d389 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:05.267588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef198346-472d-4198-bb36-5bb756e3d389 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:05.272993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97680c7-4cc1-4afd-83b3-b4372c0c15a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:05.273673 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97680c7-4cc1-4afd-83b3-b4372c0c15a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:05.273963 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97680c7-4cc1-4afd-83b3-b4372c0c15a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:05.274245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97680c7-4cc1-4afd-83b3-b4372c0c15a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:05.274972 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97680c7-4cc1-4afd-83b3-b4372c0c15a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:05.276607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97680c7-4cc1-4afd-83b3-b4372c0c15a3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:05.281325 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1001/1999] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:05.290270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 999/2000] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:05.295261 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794800aa-9bb4-4636-925e-465df4f14d3d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:05.300162 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97680c7-4cc1-4afd-83b3-b4372c0c15a3 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:05.302518 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d97680c7-4cc1-4afd-83b3-b4372c0c15a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:05.311253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794800aa-9bb4-4636-925e-465df4f14d3d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:05.317816 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f21708-fe69-4744-bb45-e5a85803a62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:05.318451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f21708-fe69-4744-bb45-e5a85803a62b admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:05.318745 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f21708-fe69-4744-bb45-e5a85803a62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:05.319712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f21708-fe69-4744-bb45-e5a85803a62b admin admin] PATH_INFO: `/v3/projects/b9e7f88f5ba34d47be9b0d5a9e419107/users/3289543ca9d64c71b231772aa629441b/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:05.337133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:05.342892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f21708-fe69-4744-bb45-e5a85803a62b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e7f88f5ba34d47be9b0d5a9e419107, user_id=3289543ca9d64c71b231772aa629441b, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:05.349313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f21708-fe69-4744-bb45-e5a85803a62b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:05.371759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1002/2001] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:05 2022] PUT /identity/v3/projects/b9e7f88f5ba34d47be9b0d5a9e419107/users/3289543ca9d64c71b231772aa629441b/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:05.376833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f21708-fe69-4744-bb45-e5a85803a62b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:05.388144 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97680c7-4cc1-4afd-83b3-b4372c0c15a3 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:05.457895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7da189-210a-437f-9b3a-69f5ba736908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:05.459586 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7da189-210a-437f-9b3a-69f5ba736908 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:05.460135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7da189-210a-437f-9b3a-69f5ba736908 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:05.460644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7da189-210a-437f-9b3a-69f5ba736908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:05.462717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f21708-fe69-4744-bb45-e5a85803a62b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:05.468234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814fad3-a233-4000-9808-7534fd811245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:05.469042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814fad3-a233-4000-9808-7534fd811245 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:05.469327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814fad3-a233-4000-9808-7534fd811245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:05.469636 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814fad3-a233-4000-9808-7534fd811245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:05.470356 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814fad3-a233-4000-9808-7534fd811245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:05.472007 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814fad3-a233-4000-9808-7534fd811245 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:05.488344 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1003/2002] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:45:05.492890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814fad3-a233-4000-9808-7534fd811245 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:05.503378 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814fad3-a233-4000-9808-7534fd811245 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:05.504829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:05.509728 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bad8d0-f0fd-4485-ae88-daf39d8333be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:05.510417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bad8d0-f0fd-4485-ae88-daf39d8333be admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:05.510700 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bad8d0-f0fd-4485-ae88-daf39d8333be admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:05.511044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bad8d0-f0fd-4485-ae88-daf39d8333be admin admin] PATH_INFO: `/v3/projects/b9e7f88f5ba34d47be9b0d5a9e419107/users/3289543ca9d64c71b231772aa629441b/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:05.513243 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bad8d0-f0fd-4485-ae88-daf39d8333be admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e7f88f5ba34d47be9b0d5a9e419107, user_id=3289543ca9d64c71b231772aa629441b, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:05.516265 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bad8d0-f0fd-4485-ae88-daf39d8333be admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:05.528442 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1004/2003] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:05 2022] PUT /identity/v3/projects/b9e7f88f5ba34d47be9b0d5a9e419107/users/3289543ca9d64c71b231772aa629441b/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:45:05.541415 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7da189-210a-437f-9b3a-69f5ba736908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:05.542880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7da189-210a-437f-9b3a-69f5ba736908 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:05.556126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1000/2004] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:45:06.517990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dfdfd-a92d-444c-bea7-551358ad649b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:06.518528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dfdfd-a92d-444c-bea7-551358ad649b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:06.518528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dfdfd-a92d-444c-bea7-551358ad649b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:06.551600 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ed9dfdfd-a92d-444c-bea7-551358ad649b None None] MFA Rules not processed for user `3289543ca9d64c71b231772aa629441b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:06.553422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ed9dfdfd-a92d-444c-bea7-551358ad649b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:06.642285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1005/2005] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:06.664624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7da189-210a-437f-9b3a-69f5ba736908 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:06.667613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6a7da189-210a-437f-9b3a-69f5ba736908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:06.686850 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7da189-210a-437f-9b3a-69f5ba736908 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:06.764993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce1c6e-e593-45fc-b1f1-46b6c577dba7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:06.765658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce1c6e-e593-45fc-b1f1-46b6c577dba7 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:06.765947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce1c6e-e593-45fc-b1f1-46b6c577dba7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:06.766222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce1c6e-e593-45fc-b1f1-46b6c577dba7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:06.826653 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce1c6e-e593-45fc-b1f1-46b6c577dba7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:06.828108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce1c6e-e593-45fc-b1f1-46b6c577dba7 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:06.837484 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1001/2006] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:07.776600 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dfdfd-a92d-444c-bea7-551358ad649b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:07.787223 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dfdfd-a92d-444c-bea7-551358ad649b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:07.790966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d69715-643e-48f8-8cb8-6c92df9569b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:07.791606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d69715-643e-48f8-8cb8-6c92df9569b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:07.791892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d69715-643e-48f8-8cb8-6c92df9569b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:07.792178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d69715-643e-48f8-8cb8-6c92df9569b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:07.794063 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-32d69715-643e-48f8-8cb8-6c92df9569b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:07.865044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:07.869299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d69715-643e-48f8-8cb8-6c92df9569b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:07.871593 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d69715-643e-48f8-8cb8-6c92df9569b9 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:07.883875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1006/2007] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:45:08.339352 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce1c6e-e593-45fc-b1f1-46b6c577dba7 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:08.350223 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce1c6e-e593-45fc-b1f1-46b6c577dba7 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:08.356962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08646132-fb49-413e-bd92-14b7708a383d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:08.357700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08646132-fb49-413e-bd92-14b7708a383d service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:08.358242 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08646132-fb49-413e-bd92-14b7708a383d service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:08.358671 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08646132-fb49-413e-bd92-14b7708a383d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:08.371004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08646132-fb49-413e-bd92-14b7708a383d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:08.372219 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08646132-fb49-413e-bd92-14b7708a383d service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:08.388578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1002/2008] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:08.400044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d69715-643e-48f8-8cb8-6c92df9569b9 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:08.415582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d69715-643e-48f8-8cb8-6c92df9569b9 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:08.420302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85a431-6070-482f-a579-1224e30865df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:08.421242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85a431-6070-482f-a579-1224e30865df service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:08.421556 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85a431-6070-482f-a579-1224e30865df service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:08.421833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85a431-6070-482f-a579-1224e30865df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:08.520725 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85a431-6070-482f-a579-1224e30865df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:08.523679 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85a431-6070-482f-a579-1224e30865df service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:08.541804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1007/2009] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:11.916812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08646132-fb49-413e-bd92-14b7708a383d service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:11.928562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08646132-fb49-413e-bd92-14b7708a383d service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:11.933323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbff6a8-fd13-4238-9d4f-57aeff6b6efb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:11.934095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbff6a8-fd13-4238-9d4f-57aeff6b6efb service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:11.934741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbff6a8-fd13-4238-9d4f-57aeff6b6efb service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:11.935404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbff6a8-fd13-4238-9d4f-57aeff6b6efb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:12.024665 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbff6a8-fd13-4238-9d4f-57aeff6b6efb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:12.026922 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbff6a8-fd13-4238-9d4f-57aeff6b6efb service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:12.041220 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1003/2010] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:18.278287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85a431-6070-482f-a579-1224e30865df service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:18.356020 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85a431-6070-482f-a579-1224e30865df service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:18.362311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bced598-0064-4823-88dd-8102c9202c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:18.363557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bced598-0064-4823-88dd-8102c9202c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:18.364050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bced598-0064-4823-88dd-8102c9202c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:18.364383 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bced598-0064-4823-88dd-8102c9202c43 admin admin] PATH_INFO: `/v3/users/920ede2286ff47df833e3055860e9c83` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:18.365889 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bced598-0064-4823-88dd-8102c9202c43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=920ede2286ff47df833e3055860e9c83)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:18.367962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bced598-0064-4823-88dd-8102c9202c43 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:18.446377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1bced598-0064-4823-88dd-8102c9202c43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920ede2286ff47df833e3055860e9c83'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:18.473373 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1bced598-0064-4823-88dd-8102c9202c43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920ede2286ff47df833e3055860e9c83'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:18.474628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1bced598-0064-4823-88dd-8102c9202c43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920ede2286ff47df833e3055860e9c83'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:18.488715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1bced598-0064-4823-88dd-8102c9202c43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920ede2286ff47df833e3055860e9c83'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:18.510619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1bced598-0064-4823-88dd-8102c9202c43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920ede2286ff47df833e3055860e9c83'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:18.519595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1008/2011] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:45:18 2022] DELETE /identity/v3/users/920ede2286ff47df833e3055860e9c83 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:18.531526 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35caccc6-be07-46ed-92dd-615cc1834585 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:18.532142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35caccc6-be07-46ed-92dd-615cc1834585 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:18.533975 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35caccc6-be07-46ed-92dd-615cc1834585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:18.596273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-35caccc6-be07-46ed-92dd-615cc1834585 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:18.598224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-35caccc6-be07-46ed-92dd-615cc1834585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:18.602576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:18.697027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1004/2012] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:18.709902 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bced598-0064-4823-88dd-8102c9202c43 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:18.711697 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1bced598-0064-4823-88dd-8102c9202c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:18.728295 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:18.790433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bced598-0064-4823-88dd-8102c9202c43 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:18.794125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0b760-1aa6-47dd-9e9a-44efc53b5a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:18.794750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0b760-1aa6-47dd-9e9a-44efc53b5a25 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:18.795026 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0b760-1aa6-47dd-9e9a-44efc53b5a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:18.795298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0b760-1aa6-47dd-9e9a-44efc53b5a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:18.805570 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0b760-1aa6-47dd-9e9a-44efc53b5a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:18.807010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0b760-1aa6-47dd-9e9a-44efc53b5a25 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:18.820968 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1009/2013] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:19.196891 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35caccc6-be07-46ed-92dd-615cc1834585 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:19.198474 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-35caccc6-be07-46ed-92dd-615cc1834585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:19.215698 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35caccc6-be07-46ed-92dd-615cc1834585 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:19.222974 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589cf38-7194-4bca-b31e-5f1b6ccf1638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:19.224360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589cf38-7194-4bca-b31e-5f1b6ccf1638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:19.224360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589cf38-7194-4bca-b31e-5f1b6ccf1638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:19.225131 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589cf38-7194-4bca-b31e-5f1b6ccf1638 admin admin] PATH_INFO: `/v3/projects/63222ebbec484fe6942b139fd31d0107` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:19.226657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589cf38-7194-4bca-b31e-5f1b6ccf1638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63222ebbec484fe6942b139fd31d0107)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:19.228748 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589cf38-7194-4bca-b31e-5f1b6ccf1638 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:19.302485 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3589cf38-7194-4bca-b31e-5f1b6ccf1638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63222ebbec484fe6942b139fd31d0107', 'request_id': 'req-3589cf38-7194-4bca-b31e-5f1b6ccf1638'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:19.309407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3589cf38-7194-4bca-b31e-5f1b6ccf1638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63222ebbec484fe6942b139fd31d0107', 'request_id': 'req-3589cf38-7194-4bca-b31e-5f1b6ccf1638'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:19.320415 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3589cf38-7194-4bca-b31e-5f1b6ccf1638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63222ebbec484fe6942b139fd31d0107', 'request_id': 'req-3589cf38-7194-4bca-b31e-5f1b6ccf1638'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:19.337824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3589cf38-7194-4bca-b31e-5f1b6ccf1638 admin admin] The token cache is being invalidate because project 63222ebbec484fe6942b139fd31d0107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:45:19.338122 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3589cf38-7194-4bca-b31e-5f1b6ccf1638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:19.338779 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3589cf38-7194-4bca-b31e-5f1b6ccf1638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:19.340314 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1005/2014] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:45:19 2022] DELETE /identity/v3/projects/63222ebbec484fe6942b139fd31d0107 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:19.344655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0b760-1aa6-47dd-9e9a-44efc53b5a25 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:19.356992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0b760-1aa6-47dd-9e9a-44efc53b5a25 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:19.437229 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f9253-3874-446c-9946-dcd36e646473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:19.449497 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f9253-3874-446c-9946-dcd36e646473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:19.449497 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f9253-3874-446c-9946-dcd36e646473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:19.449497 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f9253-3874-446c-9946-dcd36e646473 admin admin] PATH_INFO: `/v3/users/c43a69201b694568b926c110696039bc` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:19.449497 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f9253-3874-446c-9946-dcd36e646473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c43a69201b694568b926c110696039bc)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:19.449497 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f9253-3874-446c-9946-dcd36e646473 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:19.505442 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-952f9253-3874-446c-9946-dcd36e646473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c43a69201b694568b926c110696039bc'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:19.534182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-952f9253-3874-446c-9946-dcd36e646473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c43a69201b694568b926c110696039bc'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:19.535920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-952f9253-3874-446c-9946-dcd36e646473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c43a69201b694568b926c110696039bc'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:19.549545 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-952f9253-3874-446c-9946-dcd36e646473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c43a69201b694568b926c110696039bc'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:19.562156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-952f9253-3874-446c-9946-dcd36e646473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c43a69201b694568b926c110696039bc'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:19.573887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1010/2015] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:45:19 2022] DELETE /identity/v3/users/c43a69201b694568b926c110696039bc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:45:19.987433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589cf38-7194-4bca-b31e-5f1b6ccf1638 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:19.988977 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3589cf38-7194-4bca-b31e-5f1b6ccf1638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:20.002801 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589cf38-7194-4bca-b31e-5f1b6ccf1638 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:20.075446 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:20.076111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:20.076414 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:20.076657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b admin admin] PATH_INFO: `/v3/projects/0cf4305770f4438baebbc5f658903a8f` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:20.078216 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cf4305770f4438baebbc5f658903a8f)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:20.079778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:20.127879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:20.148002 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cf4305770f4438baebbc5f658903a8f', 'request_id': 'req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:20.155297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cf4305770f4438baebbc5f658903a8f', 'request_id': 'req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:20.170587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cf4305770f4438baebbc5f658903a8f', 'request_id': 'req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:20.187481 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b admin admin] The token cache is being invalidate because project 0cf4305770f4438baebbc5f658903a8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:45:20.187481 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:20.187797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:20.190206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1006/2016] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:45:19 2022] DELETE /identity/v3/projects/0cf4305770f4438baebbc5f658903a8f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:20.198910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f9253-3874-446c-9946-dcd36e646473 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:20.201209 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-952f9253-3874-446c-9946-dcd36e646473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:20.216449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f9253-3874-446c-9946-dcd36e646473 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:20.277748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:20.324415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c02917-77fc-42f8-afd1-d5a4a65c12f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:20.325090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c02917-77fc-42f8-afd1-d5a4a65c12f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:20.325373 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c02917-77fc-42f8-afd1-d5a4a65c12f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:20.325686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c02917-77fc-42f8-afd1-d5a4a65c12f8 admin admin] PATH_INFO: `/v3/users/a4b0ef2f84b84215887c79ae9c5bfcd2` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:20.326942 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c02917-77fc-42f8-afd1-d5a4a65c12f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4b0ef2f84b84215887c79ae9c5bfcd2)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:20.328896 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c02917-77fc-42f8-afd1-d5a4a65c12f8 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:20.403918 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-47c02917-77fc-42f8-afd1-d5a4a65c12f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4b0ef2f84b84215887c79ae9c5bfcd2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:20.422029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-47c02917-77fc-42f8-afd1-d5a4a65c12f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4b0ef2f84b84215887c79ae9c5bfcd2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:20.422916 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-47c02917-77fc-42f8-afd1-d5a4a65c12f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4b0ef2f84b84215887c79ae9c5bfcd2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:20.433188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-47c02917-77fc-42f8-afd1-d5a4a65c12f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4b0ef2f84b84215887c79ae9c5bfcd2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:20.441696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-47c02917-77fc-42f8-afd1-d5a4a65c12f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4b0ef2f84b84215887c79ae9c5bfcd2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:20.448893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1011/2017] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:45:20 2022] DELETE /identity/v3/users/a4b0ef2f84b84215887c79ae9c5bfcd2 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:20.813559 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:20.815769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:20.844286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d138f-5c10-47ec-a8b4-13dfda916c3b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:20.922413 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca5e7a-b168-463a-88bd-3f072e40a2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:20.923982 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca5e7a-b168-463a-88bd-3f072e40a2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:20.924538 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca5e7a-b168-463a-88bd-3f072e40a2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:20.925040 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca5e7a-b168-463a-88bd-3f072e40a2f3 admin admin] PATH_INFO: `/v3/projects/bfed0a601d1d4c5aa7e11fa7df1e613e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:20.927862 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca5e7a-b168-463a-88bd-3f072e40a2f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfed0a601d1d4c5aa7e11fa7df1e613e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:20.931679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca5e7a-b168-463a-88bd-3f072e40a2f3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:20.996361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c5ca5e7a-b168-463a-88bd-3f072e40a2f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfed0a601d1d4c5aa7e11fa7df1e613e', 'request_id': 'req-c5ca5e7a-b168-463a-88bd-3f072e40a2f3'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:21.005169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c5ca5e7a-b168-463a-88bd-3f072e40a2f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfed0a601d1d4c5aa7e11fa7df1e613e', 'request_id': 'req-c5ca5e7a-b168-463a-88bd-3f072e40a2f3'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:21.016049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c5ca5e7a-b168-463a-88bd-3f072e40a2f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfed0a601d1d4c5aa7e11fa7df1e613e', 'request_id': 'req-c5ca5e7a-b168-463a-88bd-3f072e40a2f3'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:21.030318 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c5ca5e7a-b168-463a-88bd-3f072e40a2f3 admin admin] The token cache is being invalidate because project bfed0a601d1d4c5aa7e11fa7df1e613e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:45:21.030839 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c5ca5e7a-b168-463a-88bd-3f072e40a2f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:21.031773 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c5ca5e7a-b168-463a-88bd-3f072e40a2f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:21.033423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1007/2018] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:45:20 2022] DELETE /identity/v3/projects/bfed0a601d1d4c5aa7e11fa7df1e613e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:21.040311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27650aaa-c5ac-4a77-976e-a37189149bce None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:21.040831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27650aaa-c5ac-4a77-976e-a37189149bce None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:21.041373 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27650aaa-c5ac-4a77-976e-a37189149bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:21.073084 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-27650aaa-c5ac-4a77-976e-a37189149bce None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:21.074858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-27650aaa-c5ac-4a77-976e-a37189149bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:21.152922 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1012/2019] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:21.162432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c4956-af70-4b2f-94df-8e7b47f4cbad None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:21.162907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c4956-af70-4b2f-94df-8e7b47f4cbad None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:21.163217 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c4956-af70-4b2f-94df-8e7b47f4cbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:21.189338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-730c4956-af70-4b2f-94df-8e7b47f4cbad None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:21.191836 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-730c4956-af70-4b2f-94df-8e7b47f4cbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:21.202889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1008/2020] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:21.209102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27650aaa-c5ac-4a77-976e-a37189149bce None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:21.219784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27650aaa-c5ac-4a77-976e-a37189149bce None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:21.224655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d6824-8cc5-4ef6-828f-2080e41f6968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:21.225292 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d6824-8cc5-4ef6-828f-2080e41f6968 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:21.225616 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d6824-8cc5-4ef6-828f-2080e41f6968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:21.225904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d6824-8cc5-4ef6-828f-2080e41f6968 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:21.226621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d6824-8cc5-4ef6-828f-2080e41f6968 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:45:21.226920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d6824-8cc5-4ef6-828f-2080e41f6968 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:21.228674 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d6824-8cc5-4ef6-828f-2080e41f6968 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:21.243240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1013/2021] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:45:21.253243 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c4956-af70-4b2f-94df-8e7b47f4cbad None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:21.263380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c4956-af70-4b2f-94df-8e7b47f4cbad None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:21.268163 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097198e-b75c-4943-9798-5b5eb618a0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:21.268805 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097198e-b75c-4943-9798-5b5eb618a0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:21.269636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097198e-b75c-4943-9798-5b5eb618a0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:21.269916 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097198e-b75c-4943-9798-5b5eb618a0f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:21.270601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097198e-b75c-4943-9798-5b5eb618a0f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:21.272514 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097198e-b75c-4943-9798-5b5eb618a0f8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:21.306348 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1009/2022] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:45:21 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:45:21.312462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d6824-8cc5-4ef6-828f-2080e41f6968 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:21.325113 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d6824-8cc5-4ef6-828f-2080e41f6968 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:21.374510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:21.398325 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adc0ff-9622-4d21-b5df-b197ee408c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:21.399012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adc0ff-9622-4d21-b5df-b197ee408c59 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:21.399258 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adc0ff-9622-4d21-b5df-b197ee408c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:21.399531 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adc0ff-9622-4d21-b5df-b197ee408c59 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:21.400213 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0adc0ff-9622-4d21-b5df-b197ee408c59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:21.402198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0adc0ff-9622-4d21-b5df-b197ee408c59 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:21.425891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1014/2023] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:21 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:45:21.431197 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097198e-b75c-4943-9798-5b5eb618a0f8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:21.450353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097198e-b75c-4943-9798-5b5eb618a0f8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:21.459256 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2346e0e-1b8f-4220-9617-80930daa2e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:21.460380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2346e0e-1b8f-4220-9617-80930daa2e08 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:21.460877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2346e0e-1b8f-4220-9617-80930daa2e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:21.461395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2346e0e-1b8f-4220-9617-80930daa2e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:21.462747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2346e0e-1b8f-4220-9617-80930daa2e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:21.465888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2346e0e-1b8f-4220-9617-80930daa2e08 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:21.486273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1010/2024] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:21.491197 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adc0ff-9622-4d21-b5df-b197ee408c59 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:21.505710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adc0ff-9622-4d21-b5df-b197ee408c59 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:21.511313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702f5ab-a3c8-4a24-85e4-16c2da6950f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:21.511961 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702f5ab-a3c8-4a24-85e4-16c2da6950f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:21.512246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702f5ab-a3c8-4a24-85e4-16c2da6950f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:21.512529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702f5ab-a3c8-4a24-85e4-16c2da6950f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:21.513406 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702f5ab-a3c8-4a24-85e4-16c2da6950f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:21.515063 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702f5ab-a3c8-4a24-85e4-16c2da6950f1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:21.532817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1015/2025] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:21.534153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2346e0e-1b8f-4220-9617-80930daa2e08 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:21.547049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2346e0e-1b8f-4220-9617-80930daa2e08 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:21.555728 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0fb61-a89c-4357-936a-88ab4a597b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:21.556972 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0fb61-a89c-4357-936a-88ab4a597b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:21.557407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0fb61-a89c-4357-936a-88ab4a597b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:21.557979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0fb61-a89c-4357-936a-88ab4a597b20 admin admin] PATH_INFO: `/v3/projects/8c2e98a33cba4c08a55cda8dd6ed0730/users/7cb192800160427e9951a759dd10b228/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:21.582458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:21.588868 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0fb61-a89c-4357-936a-88ab4a597b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2e98a33cba4c08a55cda8dd6ed0730, user_id=7cb192800160427e9951a759dd10b228, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:21.592543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0fb61-a89c-4357-936a-88ab4a597b20 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:21.607656 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1011/2026] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:21 2022] PUT /identity/v3/projects/8c2e98a33cba4c08a55cda8dd6ed0730/users/7cb192800160427e9951a759dd10b228/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:21.615392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8b3ae-0d9c-4327-b65a-4e8bdacc02b8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:21.615811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8b3ae-0d9c-4327-b65a-4e8bdacc02b8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:21.616177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8b3ae-0d9c-4327-b65a-4e8bdacc02b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:21.636989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e6a8b3ae-0d9c-4327-b65a-4e8bdacc02b8 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:21.638305 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e6a8b3ae-0d9c-4327-b65a-4e8bdacc02b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:21.718799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1016/2027] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:21.730214 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0fb61-a89c-4357-936a-88ab4a597b20 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:21.732013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-eab0fb61-a89c-4357-936a-88ab4a597b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:21.799717 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0fb61-a89c-4357-936a-88ab4a597b20 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:21.804580 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9007c61-8597-4b9d-a2c0-d4c163e12d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:21.805331 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007c61-8597-4b9d-a2c0-d4c163e12d57 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:21.805648 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007c61-8597-4b9d-a2c0-d4c163e12d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:21.806050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007c61-8597-4b9d-a2c0-d4c163e12d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:21.816531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9007c61-8597-4b9d-a2c0-d4c163e12d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:21.817948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9007c61-8597-4b9d-a2c0-d4c163e12d57 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:21.828624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1012/2028] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:27.446536 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becbb10-5a25-4723-989b-9317e3003727 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:27.447451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becbb10-5a25-4723-989b-9317e3003727 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:27.447886 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becbb10-5a25-4723-989b-9317e3003727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:27.492406 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4becbb10-5a25-4723-989b-9317e3003727 None None] MFA Rules not processed for user `7cb192800160427e9951a759dd10b228`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:27.494335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4becbb10-5a25-4723-989b-9317e3003727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:27.573911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1017/2029] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:27.578482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9007c61-8597-4b9d-a2c0-d4c163e12d57 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:27.591297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9007c61-8597-4b9d-a2c0-d4c163e12d57 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:27.598480 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63487a-9f21-4e44-b8de-e9f701c23086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:27.599522 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63487a-9f21-4e44-b8de-e9f701c23086 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:27.599971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63487a-9f21-4e44-b8de-e9f701c23086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:27.602848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63487a-9f21-4e44-b8de-e9f701c23086 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:27.607213 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63487a-9f21-4e44-b8de-e9f701c23086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:27.610857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63487a-9f21-4e44-b8de-e9f701c23086 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:27.662489 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1013/2030] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:45:27 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:27.667703 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4becbb10-5a25-4723-989b-9317e3003727 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:27.690672 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4becbb10-5a25-4723-989b-9317e3003727 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:27.801178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:27.828929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb618372-3202-4922-97c3-84aa4e94b976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:27.829630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb618372-3202-4922-97c3-84aa4e94b976 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:27.829914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb618372-3202-4922-97c3-84aa4e94b976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:27.830198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb618372-3202-4922-97c3-84aa4e94b976 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:27.830893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb618372-3202-4922-97c3-84aa4e94b976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:27.833317 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb618372-3202-4922-97c3-84aa4e94b976 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:27.885820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1018/2031] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:27 2022] POST /identity/v3/users => generated 292 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:27.891072 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63487a-9f21-4e44-b8de-e9f701c23086 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:27.912399 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63487a-9f21-4e44-b8de-e9f701c23086 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:27.932063 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828876d3-202c-4a5d-b045-19dabd214390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:27.933074 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828876d3-202c-4a5d-b045-19dabd214390 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:27.933432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828876d3-202c-4a5d-b045-19dabd214390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:27.933887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828876d3-202c-4a5d-b045-19dabd214390 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:27.934814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828876d3-202c-4a5d-b045-19dabd214390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:27.937597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828876d3-202c-4a5d-b045-19dabd214390 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:27.958053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1014/2032] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:27.961247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb618372-3202-4922-97c3-84aa4e94b976 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:27.983604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb618372-3202-4922-97c3-84aa4e94b976 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:27.991934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ce097-42ce-41d4-b071-c23b5d06b0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:27.992921 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ce097-42ce-41d4-b071-c23b5d06b0c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:27.993401 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ce097-42ce-41d4-b071-c23b5d06b0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:27.993693 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ce097-42ce-41d4-b071-c23b5d06b0c9 admin admin] PATH_INFO: `/v3/projects/e0b0600de07d4f01aaffe1b92d31aa49/users/987a8f8932774c3ab348360a03cf5eb8/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:28.013675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ce097-42ce-41d4-b071-c23b5d06b0c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b0600de07d4f01aaffe1b92d31aa49, user_id=987a8f8932774c3ab348360a03cf5eb8, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:28.018870 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ce097-42ce-41d4-b071-c23b5d06b0c9 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:28.046145 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1019/2033] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:27 2022] PUT /identity/v3/projects/e0b0600de07d4f01aaffe1b92d31aa49/users/987a8f8932774c3ab348360a03cf5eb8/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:28.050764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828876d3-202c-4a5d-b045-19dabd214390 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:28.070134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828876d3-202c-4a5d-b045-19dabd214390 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:28.195249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529e30a-3fe8-40cf-b669-871229208f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:28.195910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529e30a-3fe8-40cf-b669-871229208f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:28.196199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529e30a-3fe8-40cf-b669-871229208f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:28.196481 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529e30a-3fe8-40cf-b669-871229208f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:28.197219 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529e30a-3fe8-40cf-b669-871229208f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:28.198921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529e30a-3fe8-40cf-b669-871229208f6f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:28.216743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1015/2034] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:45:28.224168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ce097-42ce-41d4-b071-c23b5d06b0c9 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:28.248468 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ce097-42ce-41d4-b071-c23b5d06b0c9 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:28.257611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e4e02-8559-4f63-b80b-2c2ce834c00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:28.258825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e4e02-8559-4f63-b80b-2c2ce834c00f admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:28.262538 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e4e02-8559-4f63-b80b-2c2ce834c00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:28.263253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e4e02-8559-4f63-b80b-2c2ce834c00f admin admin] PATH_INFO: `/v3/projects/e0b0600de07d4f01aaffe1b92d31aa49/users/987a8f8932774c3ab348360a03cf5eb8/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:28.270772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e4e02-8559-4f63-b80b-2c2ce834c00f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b0600de07d4f01aaffe1b92d31aa49, user_id=987a8f8932774c3ab348360a03cf5eb8, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:28.275930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e4e02-8559-4f63-b80b-2c2ce834c00f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:28.308993 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1020/2035] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:28 2022] PUT /identity/v3/projects/e0b0600de07d4f01aaffe1b92d31aa49/users/987a8f8932774c3ab348360a03cf5eb8/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:31.670006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d1e9c-4478-4b90-b25f-70f2e9ae31b8 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:31.670555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d1e9c-4478-4b90-b25f-70f2e9ae31b8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:31.670662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d1e9c-4478-4b90-b25f-70f2e9ae31b8 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:31.673342 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1016/2036] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 3 10:45:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:45:31.686350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda58f5a-6c26-4388-b0bb-0762aed16952 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:31.687299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda58f5a-6c26-4388-b0bb-0762aed16952 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:31.687859 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda58f5a-6c26-4388-b0bb-0762aed16952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:31.756165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-bda58f5a-6c26-4388-b0bb-0762aed16952 None None] MFA Rules not processed for user `f2c5536eddd346d692822a2f3befb48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:31.759841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bda58f5a-6c26-4388-b0bb-0762aed16952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:31.849704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d1e9c-4478-4b90-b25f-70f2e9ae31b8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:31.852907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9b8d1e9c-4478-4b90-b25f-70f2e9ae31b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:32.026884 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1021/2037] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 3 10:45:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:45:32.055670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:32.055909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda58f5a-6c26-4388-b0bb-0762aed16952 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:32.058455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bda58f5a-6c26-4388-b0bb-0762aed16952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:32.065677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d1e9c-4478-4b90-b25f-70f2e9ae31b8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:32.074349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109ca10-b7a7-4122-895c-9c5dbbe24a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:32.075466 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109ca10-b7a7-4122-895c-9c5dbbe24a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:32.075995 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109ca10-b7a7-4122-895c-9c5dbbe24a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:32.076490 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109ca10-b7a7-4122-895c-9c5dbbe24a7b admin admin] PATH_INFO: `/v3/users/d560ec3b091c4baab93c4abb5fabd287` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:32.078370 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109ca10-b7a7-4122-895c-9c5dbbe24a7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d560ec3b091c4baab93c4abb5fabd287)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:32.084197 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109ca10-b7a7-4122-895c-9c5dbbe24a7b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:32.104652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda58f5a-6c26-4388-b0bb-0762aed16952 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:32.139004 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36221da7-6664-4f12-9356-5855e7056c00 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2c5536eddd346d692822a2f3befb48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:32.140416 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36221da7-6664-4f12-9356-5855e7056c00 service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:32.140939 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36221da7-6664-4f12-9356-5855e7056c00 service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:32.141614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36221da7-6664-4f12-9356-5855e7056c00 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:32.197813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36221da7-6664-4f12-9356-5855e7056c00 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:32.201190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1109ca10-b7a7-4122-895c-9c5dbbe24a7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd560ec3b091c4baab93c4abb5fabd287'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:32.201453 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36221da7-6664-4f12-9356-5855e7056c00 service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:32.223324 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1109ca10-b7a7-4122-895c-9c5dbbe24a7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd560ec3b091c4baab93c4abb5fabd287'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:32.229648 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1022/2038] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:32.255633 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1109ca10-b7a7-4122-895c-9c5dbbe24a7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd560ec3b091c4baab93c4abb5fabd287'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:32.256903 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1109ca10-b7a7-4122-895c-9c5dbbe24a7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd560ec3b091c4baab93c4abb5fabd287'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:32.258628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1109ca10-b7a7-4122-895c-9c5dbbe24a7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd560ec3b091c4baab93c4abb5fabd287'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:32.334983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1017/2039] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:45:31 2022] DELETE /identity/v3/users/d560ec3b091c4baab93c4abb5fabd287 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:45:33.322560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36221da7-6664-4f12-9356-5855e7056c00 service placement] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:33.358807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36221da7-6664-4f12-9356-5855e7056c00 service placement] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:33.464584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ba30b-0f88-474a-b88a-e26cd8ecfb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:33.466251 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ba30b-0f88-474a-b88a-e26cd8ecfb35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:33.469544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ba30b-0f88-474a-b88a-e26cd8ecfb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:33.470183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ba30b-0f88-474a-b88a-e26cd8ecfb35 admin admin] PATH_INFO: `/v3/projects/ab735db52f2e420e9aeba9387484dd07` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:33.474995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ba30b-0f88-474a-b88a-e26cd8ecfb35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab735db52f2e420e9aeba9387484dd07)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:33.477283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ba30b-0f88-474a-b88a-e26cd8ecfb35 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:33.589098 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:33.642286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d88ba30b-0f88-474a-b88a-e26cd8ecfb35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab735db52f2e420e9aeba9387484dd07', 'request_id': 'req-d88ba30b-0f88-474a-b88a-e26cd8ecfb35'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:33.666347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d88ba30b-0f88-474a-b88a-e26cd8ecfb35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab735db52f2e420e9aeba9387484dd07', 'request_id': 'req-d88ba30b-0f88-474a-b88a-e26cd8ecfb35'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:33.701983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d88ba30b-0f88-474a-b88a-e26cd8ecfb35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab735db52f2e420e9aeba9387484dd07', 'request_id': 'req-d88ba30b-0f88-474a-b88a-e26cd8ecfb35'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:33.733053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d88ba30b-0f88-474a-b88a-e26cd8ecfb35 admin admin] The token cache is being invalidate because project ab735db52f2e420e9aeba9387484dd07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:45:33.734520 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d88ba30b-0f88-474a-b88a-e26cd8ecfb35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:33.743301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d88ba30b-0f88-474a-b88a-e26cd8ecfb35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:33.747660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1023/2040] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:45:33 2022] DELETE /identity/v3/projects/ab735db52f2e420e9aeba9387484dd07 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:33.755998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d012e-fc0a-41c7-81f0-854956f90dc7 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:33.755998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d012e-fc0a-41c7-81f0-854956f90dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:33.755998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d012e-fc0a-41c7-81f0-854956f90dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:33.811907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-882d012e-fc0a-41c7-81f0-854956f90dc7 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:33.816417 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-882d012e-fc0a-41c7-81f0-854956f90dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:33.973021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1018/2041] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:33.981004 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eee3e6-8357-4c95-88b1-5b34bfda1a5c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:33.982157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eee3e6-8357-4c95-88b1-5b34bfda1a5c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:33.982738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eee3e6-8357-4c95-88b1-5b34bfda1a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:34.035115 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-46eee3e6-8357-4c95-88b1-5b34bfda1a5c None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:34.042285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-46eee3e6-8357-4c95-88b1-5b34bfda1a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:34.062549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1024/2042] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:45:34.066844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d012e-fc0a-41c7-81f0-854956f90dc7 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:34.090494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d012e-fc0a-41c7-81f0-854956f90dc7 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:34.103076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c6c6-90e0-49a6-abfb-96ce38617c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:34.105549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c6c6-90e0-49a6-abfb-96ce38617c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:34.106670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c6c6-90e0-49a6-abfb-96ce38617c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:34.107253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c6c6-90e0-49a6-abfb-96ce38617c1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:34.110746 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4c6c6-90e0-49a6-abfb-96ce38617c1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:45:34.113058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4c6c6-90e0-49a6-abfb-96ce38617c1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:34.119570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4c6c6-90e0-49a6-abfb-96ce38617c1d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:34.152677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1019/2043] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:45:34.158962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eee3e6-8357-4c95-88b1-5b34bfda1a5c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:34.174790 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eee3e6-8357-4c95-88b1-5b34bfda1a5c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:34.198871 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412047b-2110-4dbb-8962-addbbbaad2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:34.199712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412047b-2110-4dbb-8962-addbbbaad2f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:34.199814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412047b-2110-4dbb-8962-addbbbaad2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:34.200132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412047b-2110-4dbb-8962-addbbbaad2f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:34.201118 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412047b-2110-4dbb-8962-addbbbaad2f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:34.203280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412047b-2110-4dbb-8962-addbbbaad2f3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:34.280314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1025/2044] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:45:34 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:34.283909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c6c6-90e0-49a6-abfb-96ce38617c1d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:34.307657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c6c6-90e0-49a6-abfb-96ce38617c1d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:34.430283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412047b-2110-4dbb-8962-addbbbaad2f3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:34.432763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3412047b-2110-4dbb-8962-addbbbaad2f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:34.444653 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124f54a-4998-41c4-b730-db4affde9e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:34.446190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124f54a-4998-41c4-b730-db4affde9e1b admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:34.446782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124f54a-4998-41c4-b730-db4affde9e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:34.447462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124f54a-4998-41c4-b730-db4affde9e1b admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:34.448764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124f54a-4998-41c4-b730-db4affde9e1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:34.452698 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124f54a-4998-41c4-b730-db4affde9e1b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:34.469781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:34.501652 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1020/2045] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:34 2022] POST /identity/v3/users => generated 284 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:34.501768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412047b-2110-4dbb-8962-addbbbaad2f3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:34.508533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124f54a-4998-41c4-b730-db4affde9e1b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:34.521114 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb9943-abae-49bc-97fd-0bcbc26cfe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:34.521959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb9943-abae-49bc-97fd-0bcbc26cfe05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:34.522309 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb9943-abae-49bc-97fd-0bcbc26cfe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:34.522711 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb9943-abae-49bc-97fd-0bcbc26cfe05 admin admin] PATH_INFO: `/v3/users/b7fb2b5c4c934d229033282b5a813d09` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:34.534247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb9943-abae-49bc-97fd-0bcbc26cfe05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7fb2b5c4c934d229033282b5a813d09)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:34.538176 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb9943-abae-49bc-97fd-0bcbc26cfe05 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:34.543510 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124f54a-4998-41c4-b730-db4affde9e1b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:34.559952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f04c7-eac1-466e-9ce7-6af5312f9d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:34.560611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f04c7-eac1-466e-9ce7-6af5312f9d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:34.560887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f04c7-eac1-466e-9ce7-6af5312f9d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:34.561160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f04c7-eac1-466e-9ce7-6af5312f9d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:34.561947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f04c7-eac1-466e-9ce7-6af5312f9d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:34.564422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f04c7-eac1-466e-9ce7-6af5312f9d77 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:34.582263 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1021/2046] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:45:34.587819 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f04c7-eac1-466e-9ce7-6af5312f9d77 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:34.606999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f04c7-eac1-466e-9ce7-6af5312f9d77 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:34.633607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b76ae-46fc-41fd-b990-e05ff4055959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:34.634650 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b76ae-46fc-41fd-b990-e05ff4055959 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:34.635965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b76ae-46fc-41fd-b990-e05ff4055959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:34.636291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b76ae-46fc-41fd-b990-e05ff4055959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:34.637496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b76ae-46fc-41fd-b990-e05ff4055959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:34.639989 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b76ae-46fc-41fd-b990-e05ff4055959 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:34.640719 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-59eb9943-abae-49bc-97fd-0bcbc26cfe05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7fb2b5c4c934d229033282b5a813d09'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:34.663163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-59eb9943-abae-49bc-97fd-0bcbc26cfe05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7fb2b5c4c934d229033282b5a813d09'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:34.664170 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-59eb9943-abae-49bc-97fd-0bcbc26cfe05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7fb2b5c4c934d229033282b5a813d09'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:34.672342 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1022/2047] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:45:34.674013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b76ae-46fc-41fd-b990-e05ff4055959 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:34.677494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-719b76ae-46fc-41fd-b990-e05ff4055959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:34.690540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-59eb9943-abae-49bc-97fd-0bcbc26cfe05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7fb2b5c4c934d229033282b5a813d09'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:34.698653 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b76ae-46fc-41fd-b990-e05ff4055959 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:34.700986 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-59eb9943-abae-49bc-97fd-0bcbc26cfe05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7fb2b5c4c934d229033282b5a813d09'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:34.704119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1155e06-bbd2-4005-9c40-2e2c9f80bf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:34.704810 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1155e06-bbd2-4005-9c40-2e2c9f80bf24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:34.705227 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1155e06-bbd2-4005-9c40-2e2c9f80bf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:34.705593 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1155e06-bbd2-4005-9c40-2e2c9f80bf24 admin admin] PATH_INFO: `/v3/projects/06e59c62f01f4f0383b0a9d75b63b7b2/users/023bc40a6c274d0c80e0e5a469a1980f/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:34.710530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1026/2048] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:45:34 2022] DELETE /identity/v3/users/b7fb2b5c4c934d229033282b5a813d09 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:45:34.727141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1155e06-bbd2-4005-9c40-2e2c9f80bf24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e59c62f01f4f0383b0a9d75b63b7b2, user_id=023bc40a6c274d0c80e0e5a469a1980f, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:34.731783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1155e06-bbd2-4005-9c40-2e2c9f80bf24 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:34.756572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1023/2049] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:34 2022] PUT /identity/v3/projects/06e59c62f01f4f0383b0a9d75b63b7b2/users/023bc40a6c274d0c80e0e5a469a1980f/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:45:34.768433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843aa77-31b9-4d87-8a38-f98bf09d301f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:34.771954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843aa77-31b9-4d87-8a38-f98bf09d301f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:34.772398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843aa77-31b9-4d87-8a38-f98bf09d301f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:34.829324 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4843aa77-31b9-4d87-8a38-f98bf09d301f None None] MFA Rules not processed for user `023bc40a6c274d0c80e0e5a469a1980f`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:34.830923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4843aa77-31b9-4d87-8a38-f98bf09d301f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:34.901697 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:34.936468 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1027/2050] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:34.950348 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1155e06-bbd2-4005-9c40-2e2c9f80bf24 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:34.952508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e1155e06-bbd2-4005-9c40-2e2c9f80bf24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:35.007018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843aa77-31b9-4d87-8a38-f98bf09d301f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:35.009504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4843aa77-31b9-4d87-8a38-f98bf09d301f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:35.084526 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1155e06-bbd2-4005-9c40-2e2c9f80bf24 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:35.088972 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfe3a7-a715-4643-a4ef-0304a6b9e822 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:35.089848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfe3a7-a715-4643-a4ef-0304a6b9e822 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:35.090239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfe3a7-a715-4643-a4ef-0304a6b9e822 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:35.090626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfe3a7-a715-4643-a4ef-0304a6b9e822 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:35.104459 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfe3a7-a715-4643-a4ef-0304a6b9e822 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:35.107241 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfe3a7-a715-4643-a4ef-0304a6b9e822 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:35.125321 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1024/2051] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:35.127593 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfe3a7-a715-4643-a4ef-0304a6b9e822 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:35.130182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-34dfe3a7-a715-4643-a4ef-0304a6b9e822 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:35.143149 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843aa77-31b9-4d87-8a38-f98bf09d301f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:35.151895 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f2211-127e-4062-a677-68e6c083f2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:35.152584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f2211-127e-4062-a677-68e6c083f2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:35.152870 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f2211-127e-4062-a677-68e6c083f2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:35.153143 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f2211-127e-4062-a677-68e6c083f2f1 admin admin] PATH_INFO: `/v3/projects/c517457e664a42118ea0a0cdb1523dd5` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:35.154489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f2211-127e-4062-a677-68e6c083f2f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c517457e664a42118ea0a0cdb1523dd5)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:35.156618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f2211-127e-4062-a677-68e6c083f2f1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:35.161040 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfe3a7-a715-4643-a4ef-0304a6b9e822 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:35.176569 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3486e6-48b2-4f23-9dac-3d9513c1d95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:35.178564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3486e6-48b2-4f23-9dac-3d9513c1d95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:35.179483 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3486e6-48b2-4f23-9dac-3d9513c1d95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:35.181530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3486e6-48b2-4f23-9dac-3d9513c1d95d admin admin] PATH_INFO: `/v3/users/b05ed0dd293948c2a9834075222a1137` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:35.185749 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3486e6-48b2-4f23-9dac-3d9513c1d95d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b05ed0dd293948c2a9834075222a1137)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:35.192235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3486e6-48b2-4f23-9dac-3d9513c1d95d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:35.225075 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-792f2211-127e-4062-a677-68e6c083f2f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c517457e664a42118ea0a0cdb1523dd5', 'request_id': 'req-792f2211-127e-4062-a677-68e6c083f2f1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:35.275780 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-792f2211-127e-4062-a677-68e6c083f2f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c517457e664a42118ea0a0cdb1523dd5', 'request_id': 'req-792f2211-127e-4062-a677-68e6c083f2f1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:35.301693 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-792f2211-127e-4062-a677-68e6c083f2f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c517457e664a42118ea0a0cdb1523dd5', 'request_id': 'req-792f2211-127e-4062-a677-68e6c083f2f1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:35.319177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ca3486e6-48b2-4f23-9dac-3d9513c1d95d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b05ed0dd293948c2a9834075222a1137'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:35.334157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-792f2211-127e-4062-a677-68e6c083f2f1 admin admin] The token cache is being invalidate because project c517457e664a42118ea0a0cdb1523dd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:45:35.334477 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-792f2211-127e-4062-a677-68e6c083f2f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:35.336440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-792f2211-127e-4062-a677-68e6c083f2f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:35.336776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ca3486e6-48b2-4f23-9dac-3d9513c1d95d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b05ed0dd293948c2a9834075222a1137'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:35.345542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1028/2052] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:45:35 2022] DELETE /identity/v3/projects/c517457e664a42118ea0a0cdb1523dd5 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:35.347564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f2211-127e-4062-a677-68e6c083f2f1 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:35.352453 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-792f2211-127e-4062-a677-68e6c083f2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:35.362558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ca3486e6-48b2-4f23-9dac-3d9513c1d95d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b05ed0dd293948c2a9834075222a1137'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:35.365743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ca3486e6-48b2-4f23-9dac-3d9513c1d95d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b05ed0dd293948c2a9834075222a1137'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:35.368350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ca3486e6-48b2-4f23-9dac-3d9513c1d95d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b05ed0dd293948c2a9834075222a1137'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:35.414708 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1025/2053] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:45:35 2022] DELETE /identity/v3/users/b05ed0dd293948c2a9834075222a1137 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:35.420012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82636d72-d9df-4db4-b69b-901163a39baf None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:35.420936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82636d72-d9df-4db4-b69b-901163a39baf None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:35.424743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82636d72-d9df-4db4-b69b-901163a39baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:35.458772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-82636d72-d9df-4db4-b69b-901163a39baf None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:35.460783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-82636d72-d9df-4db4-b69b-901163a39baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:35.490194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:35.537572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f2211-127e-4062-a677-68e6c083f2f1 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:35.575914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:35.641567 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7cc6e-2f56-4a04-8ed3-5afba0687d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:35.642295 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7cc6e-2f56-4a04-8ed3-5afba0687d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:35.642659 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7cc6e-2f56-4a04-8ed3-5afba0687d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:35.642949 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7cc6e-2f56-4a04-8ed3-5afba0687d05 admin admin] PATH_INFO: `/v3/users/3289543ca9d64c71b231772aa629441b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:35.649645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7cc6e-2f56-4a04-8ed3-5afba0687d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3289543ca9d64c71b231772aa629441b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:35.649645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7cc6e-2f56-4a04-8ed3-5afba0687d05 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:35.675519 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1026/2054] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:35.680244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82636d72-d9df-4db4-b69b-901163a39baf None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:35.688361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-82636d72-d9df-4db4-b69b-901163a39baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:35.755211 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5ef7cc6e-2f56-4a04-8ed3-5afba0687d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3289543ca9d64c71b231772aa629441b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:35.790834 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5ef7cc6e-2f56-4a04-8ed3-5afba0687d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3289543ca9d64c71b231772aa629441b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:35.792738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5ef7cc6e-2f56-4a04-8ed3-5afba0687d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3289543ca9d64c71b231772aa629441b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:35.820537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5ef7cc6e-2f56-4a04-8ed3-5afba0687d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3289543ca9d64c71b231772aa629441b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:35.843358 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5ef7cc6e-2f56-4a04-8ed3-5afba0687d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3289543ca9d64c71b231772aa629441b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:35.864631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1029/2055] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:45:35 2022] DELETE /identity/v3/users/3289543ca9d64c71b231772aa629441b => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:45:35.890205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82636d72-d9df-4db4-b69b-901163a39baf None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:36.013346 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfaa6ca-6684-4d09-9912-23fcdf8c9b74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:36.014007 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaa6ca-6684-4d09-9912-23fcdf8c9b74 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:36.014286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaa6ca-6684-4d09-9912-23fcdf8c9b74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:36.014561 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaa6ca-6684-4d09-9912-23fcdf8c9b74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:36.016451 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ebfaa6ca-6684-4d09-9912-23fcdf8c9b74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:36.199322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfaa6ca-6684-4d09-9912-23fcdf8c9b74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:36.209079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfaa6ca-6684-4d09-9912-23fcdf8c9b74 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:36.241857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1027/2056] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:36.508657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7cc6e-2f56-4a04-8ed3-5afba0687d05 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:36.510232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5ef7cc6e-2f56-4a04-8ed3-5afba0687d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:36.530150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7cc6e-2f56-4a04-8ed3-5afba0687d05 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:36.551666 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548088f2-c55d-4d26-b4a9-826e6d4592b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:36.552697 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548088f2-c55d-4d26-b4a9-826e6d4592b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:36.552697 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548088f2-c55d-4d26-b4a9-826e6d4592b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:36.552982 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548088f2-c55d-4d26-b4a9-826e6d4592b6 admin admin] PATH_INFO: `/v3/projects/b9e7f88f5ba34d47be9b0d5a9e419107` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:36.556877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548088f2-c55d-4d26-b4a9-826e6d4592b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e7f88f5ba34d47be9b0d5a9e419107)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:36.559088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548088f2-c55d-4d26-b4a9-826e6d4592b6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:36.657322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-548088f2-c55d-4d26-b4a9-826e6d4592b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e7f88f5ba34d47be9b0d5a9e419107', 'request_id': 'req-548088f2-c55d-4d26-b4a9-826e6d4592b6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:36.670180 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-548088f2-c55d-4d26-b4a9-826e6d4592b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e7f88f5ba34d47be9b0d5a9e419107', 'request_id': 'req-548088f2-c55d-4d26-b4a9-826e6d4592b6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:36.708120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-548088f2-c55d-4d26-b4a9-826e6d4592b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e7f88f5ba34d47be9b0d5a9e419107', 'request_id': 'req-548088f2-c55d-4d26-b4a9-826e6d4592b6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:36.746843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-548088f2-c55d-4d26-b4a9-826e6d4592b6 admin admin] The token cache is being invalidate because project b9e7f88f5ba34d47be9b0d5a9e419107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:45:36.747880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-548088f2-c55d-4d26-b4a9-826e6d4592b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:36.748056 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-548088f2-c55d-4d26-b4a9-826e6d4592b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:36.764176 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1030/2057] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:45:36 2022] DELETE /identity/v3/projects/b9e7f88f5ba34d47be9b0d5a9e419107 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:45:36.767409 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd3f9b-9c2f-4a4f-baab-c81ea7ca5d33 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:36.768058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd3f9b-9c2f-4a4f-baab-c81ea7ca5d33 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:36.768626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd3f9b-9c2f-4a4f-baab-c81ea7ca5d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:36.824721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-04fd3f9b-9c2f-4a4f-baab-c81ea7ca5d33 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:36.829189 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-04fd3f9b-9c2f-4a4f-baab-c81ea7ca5d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:36.910381 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:37.060851 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1028/2058] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:37.071260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14abbf-747a-4bb4-af85-4dfe3f84ee2b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:37.071528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14abbf-747a-4bb4-af85-4dfe3f84ee2b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:37.071815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14abbf-747a-4bb4-af85-4dfe3f84ee2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:37.096319 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd3f9b-9c2f-4a4f-baab-c81ea7ca5d33 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:37.104014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-04fd3f9b-9c2f-4a4f-baab-c81ea7ca5d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:37.114123 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5a14abbf-747a-4bb4-af85-4dfe3f84ee2b None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:37.119797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5a14abbf-747a-4bb4-af85-4dfe3f84ee2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:37.146978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1031/2059] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:37.151253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14abbf-747a-4bb4-af85-4dfe3f84ee2b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:37.156276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd3f9b-9c2f-4a4f-baab-c81ea7ca5d33 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:37.170238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14abbf-747a-4bb4-af85-4dfe3f84ee2b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:37.175028 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e8e8b-9402-4da5-a42f-dca3c96236ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:37.178082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e8e8b-9402-4da5-a42f-dca3c96236ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:37.179184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e8e8b-9402-4da5-a42f-dca3c96236ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:37.179184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e8e8b-9402-4da5-a42f-dca3c96236ba admin admin] PATH_INFO: `/v3/projects/661c413d13a945e390de4172f249cdad` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:37.183997 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e8e8b-9402-4da5-a42f-dca3c96236ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=661c413d13a945e390de4172f249cdad)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:37.186117 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e8e8b-9402-4da5-a42f-dca3c96236ba admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:37.190108 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5b989-e80a-4cf4-8703-37a0aceeea47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:37.191152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5b989-e80a-4cf4-8703-37a0aceeea47 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:37.191590 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5b989-e80a-4cf4-8703-37a0aceeea47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:37.192202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5b989-e80a-4cf4-8703-37a0aceeea47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:37.193105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5b989-e80a-4cf4-8703-37a0aceeea47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:45:37.193598 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5b989-e80a-4cf4-8703-37a0aceeea47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:37.195785 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5b989-e80a-4cf4-8703-37a0aceeea47 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:37.214213 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1032/2060] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:45:37.220556 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5b989-e80a-4cf4-8703-37a0aceeea47 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:37.255139 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5b989-e80a-4cf4-8703-37a0aceeea47 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:37.272771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5df82-fbdb-457b-8778-108d6f9f54f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:37.274484 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5df82-fbdb-457b-8778-108d6f9f54f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:37.274963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5df82-fbdb-457b-8778-108d6f9f54f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:37.276892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5df82-fbdb-457b-8778-108d6f9f54f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:37.279286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5df82-fbdb-457b-8778-108d6f9f54f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:37.283825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5df82-fbdb-457b-8778-108d6f9f54f5 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:37.300885 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:37.348379 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1033/2061] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:45:37 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:37.354639 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5df82-fbdb-457b-8778-108d6f9f54f5 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:37.403864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-399e8e8b-9402-4da5-a42f-dca3c96236ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '661c413d13a945e390de4172f249cdad', 'request_id': 'req-399e8e8b-9402-4da5-a42f-dca3c96236ba'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:37.434693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-399e8e8b-9402-4da5-a42f-dca3c96236ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '661c413d13a945e390de4172f249cdad', 'request_id': 'req-399e8e8b-9402-4da5-a42f-dca3c96236ba'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:37.452829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-399e8e8b-9402-4da5-a42f-dca3c96236ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '661c413d13a945e390de4172f249cdad', 'request_id': 'req-399e8e8b-9402-4da5-a42f-dca3c96236ba'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:37.489089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-399e8e8b-9402-4da5-a42f-dca3c96236ba admin admin] The token cache is being invalidate because project 661c413d13a945e390de4172f249cdad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:45:37.489089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-399e8e8b-9402-4da5-a42f-dca3c96236ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:37.492626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-399e8e8b-9402-4da5-a42f-dca3c96236ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:37.495962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1029/2062] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:45:37 2022] DELETE /identity/v3/projects/661c413d13a945e390de4172f249cdad => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:37.500358 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e8e8b-9402-4da5-a42f-dca3c96236ba admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:37.502909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-399e8e8b-9402-4da5-a42f-dca3c96236ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:37.528843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5df82-fbdb-457b-8778-108d6f9f54f5 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:37.551655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db6f68-a9b9-464c-9cb8-c8ebca73da17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:37.551655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db6f68-a9b9-464c-9cb8-c8ebca73da17 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:37.553142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db6f68-a9b9-464c-9cb8-c8ebca73da17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:37.553142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db6f68-a9b9-464c-9cb8-c8ebca73da17 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:37.553142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db6f68-a9b9-464c-9cb8-c8ebca73da17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:37.555057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db6f68-a9b9-464c-9cb8-c8ebca73da17 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:37.606968 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1034/2063] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:37 2022] POST /identity/v3/users => generated 287 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:37.610334 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db6f68-a9b9-464c-9cb8-c8ebca73da17 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:37.615816 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-09db6f68-a9b9-464c-9cb8-c8ebca73da17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:37.673785 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db6f68-a9b9-464c-9cb8-c8ebca73da17 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:37.707633 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d4cbc-9682-4e28-b040-ef8a131f8671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:37.709578 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d4cbc-9682-4e28-b040-ef8a131f8671 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:37.709871 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d4cbc-9682-4e28-b040-ef8a131f8671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:37.710156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d4cbc-9682-4e28-b040-ef8a131f8671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:37.714680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d4cbc-9682-4e28-b040-ef8a131f8671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:37.719505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d4cbc-9682-4e28-b040-ef8a131f8671 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:37.730943 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e8e8b-9402-4da5-a42f-dca3c96236ba admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:37.739955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6edf1-3dc0-4c99-aa49-54fe334d273c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:37.740775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6edf1-3dc0-4c99-aa49-54fe334d273c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:37.741094 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6edf1-3dc0-4c99-aa49-54fe334d273c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:37.741912 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6edf1-3dc0-4c99-aa49-54fe334d273c admin admin] PATH_INFO: `/v3/users/1c09fb8f708142099a6805e61df0a8fc` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:37.743590 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6edf1-3dc0-4c99-aa49-54fe334d273c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c09fb8f708142099a6805e61df0a8fc)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:37.746613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6edf1-3dc0-4c99-aa49-54fe334d273c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:37.754630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1035/2064] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:37.765087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d4cbc-9682-4e28-b040-ef8a131f8671 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:37.794493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d4cbc-9682-4e28-b040-ef8a131f8671 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:37.805424 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54018781-f72b-460a-a1af-eab22ca8ae3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:37.814269 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54018781-f72b-460a-a1af-eab22ca8ae3f admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:37.814269 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54018781-f72b-460a-a1af-eab22ca8ae3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:37.814269 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54018781-f72b-460a-a1af-eab22ca8ae3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:37.814269 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54018781-f72b-460a-a1af-eab22ca8ae3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:37.828726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54018781-f72b-460a-a1af-eab22ca8ae3f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:37.867166 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5bf6edf1-3dc0-4c99-aa49-54fe334d273c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c09fb8f708142099a6805e61df0a8fc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:37.868676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1036/2065] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:37.872323 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051e06d-08d2-443e-9987-cd10a8c0431c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:37.878856 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:37.885974 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051e06d-08d2-443e-9987-cd10a8c0431c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:37.886281 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051e06d-08d2-443e-9987-cd10a8c0431c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:37.895491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5bf6edf1-3dc0-4c99-aa49-54fe334d273c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c09fb8f708142099a6805e61df0a8fc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:37.919249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5bf6edf1-3dc0-4c99-aa49-54fe334d273c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c09fb8f708142099a6805e61df0a8fc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:37.923226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5bf6edf1-3dc0-4c99-aa49-54fe334d273c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c09fb8f708142099a6805e61df0a8fc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:37.923226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5bf6edf1-3dc0-4c99-aa49-54fe334d273c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c09fb8f708142099a6805e61df0a8fc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:37.929842 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c051e06d-08d2-443e-9987-cd10a8c0431c None None] MFA Rules not processed for user `987a8f8932774c3ab348360a03cf5eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:37.931436 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c051e06d-08d2-443e-9987-cd10a8c0431c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:37.961627 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1030/2066] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:45:37 2022] DELETE /identity/v3/users/1c09fb8f708142099a6805e61df0a8fc => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:37.964968 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6edf1-3dc0-4c99-aa49-54fe334d273c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:37.968271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5bf6edf1-3dc0-4c99-aa49-54fe334d273c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:38.047104 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1037/2067] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:38.062795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051e06d-08d2-443e-9987-cd10a8c0431c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:38.068975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c051e06d-08d2-443e-9987-cd10a8c0431c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:38.181533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:38.245807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6edf1-3dc0-4c99-aa49-54fe334d273c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:38.254339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183e7a4-60b0-4491-8d95-0e1f5ddc45f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:38.255569 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183e7a4-60b0-4491-8d95-0e1f5ddc45f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:38.256085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183e7a4-60b0-4491-8d95-0e1f5ddc45f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:38.256729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183e7a4-60b0-4491-8d95-0e1f5ddc45f8 admin admin] PATH_INFO: `/v3/projects/790e59abcdf14039869204561c535f75/users/998108a1ef4b476c8aedaab1aa8ead06/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:38.257922 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051e06d-08d2-443e-9987-cd10a8c0431c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:38.293836 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183e7a4-60b0-4491-8d95-0e1f5ddc45f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=790e59abcdf14039869204561c535f75, user_id=998108a1ef4b476c8aedaab1aa8ead06, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:38.299851 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183e7a4-60b0-4491-8d95-0e1f5ddc45f8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:38.341392 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1031/2068] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:37 2022] PUT /identity/v3/projects/790e59abcdf14039869204561c535f75/users/998108a1ef4b476c8aedaab1aa8ead06/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:38.343017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd2435-789e-4293-a086-b5996e2e7561 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:38.343323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd2435-789e-4293-a086-b5996e2e7561 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:38.343602 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd2435-789e-4293-a086-b5996e2e7561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:38.383740 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-cebd2435-789e-4293-a086-b5996e2e7561 None None] MFA Rules not processed for user `998108a1ef4b476c8aedaab1aa8ead06`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:38.385393 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cebd2435-789e-4293-a086-b5996e2e7561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:38.420989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386b508-ac05-47d3-9d4b-8ad03e9cc41c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:38.421724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386b508-ac05-47d3-9d4b-8ad03e9cc41c service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:38.422020 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386b508-ac05-47d3-9d4b-8ad03e9cc41c service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:38.422413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386b508-ac05-47d3-9d4b-8ad03e9cc41c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:38.424672 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c386b508-ac05-47d3-9d4b-8ad03e9cc41c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:38.542005 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1032/2069] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:38.560356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd2435-789e-4293-a086-b5996e2e7561 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:38.562635 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cebd2435-789e-4293-a086-b5996e2e7561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:38.584839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386b508-ac05-47d3-9d4b-8ad03e9cc41c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:38.587846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386b508-ac05-47d3-9d4b-8ad03e9cc41c service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:38.607287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1038/2070] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:45:38.667598 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:38.687662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd2435-789e-4293-a086-b5996e2e7561 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:38.693782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdfe64-92d4-4b66-97c0-baf39c7dc7cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:38.694581 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdfe64-92d4-4b66-97c0-baf39c7dc7cb service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:38.696282 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdfe64-92d4-4b66-97c0-baf39c7dc7cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:38.696568 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdfe64-92d4-4b66-97c0-baf39c7dc7cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:38.739950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdfe64-92d4-4b66-97c0-baf39c7dc7cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:38.742229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdfe64-92d4-4b66-97c0-baf39c7dc7cb service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:38.759606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386b508-ac05-47d3-9d4b-8ad03e9cc41c service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:38.765941 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c386b508-ac05-47d3-9d4b-8ad03e9cc41c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:38.767814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1033/2071] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:45:38.794907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdfe64-92d4-4b66-97c0-baf39c7dc7cb service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:38.796494 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386b508-ac05-47d3-9d4b-8ad03e9cc41c service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:38.947101 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad16c6-591a-4ef8-8201-5560f45e4e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:38.948175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad16c6-591a-4ef8-8201-5560f45e4e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:38.948468 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad16c6-591a-4ef8-8201-5560f45e4e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:38.948746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad16c6-591a-4ef8-8201-5560f45e4e12 admin admin] PATH_INFO: `/v3/projects/b4019536f1da486cbadd801f3b939585` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:38.950124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad16c6-591a-4ef8-8201-5560f45e4e12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4019536f1da486cbadd801f3b939585)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:38.952112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad16c6-591a-4ef8-8201-5560f45e4e12 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:39.018428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdfe64-92d4-4b66-97c0-baf39c7dc7cb service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:39.025119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c1d65-89ae-4e9c-8388-73eefcdf40dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:39.026194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c1d65-89ae-4e9c-8388-73eefcdf40dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:39.026492 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c1d65-89ae-4e9c-8388-73eefcdf40dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:39.026772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c1d65-89ae-4e9c-8388-73eefcdf40dc admin admin] PATH_INFO: `/v3/users/998108a1ef4b476c8aedaab1aa8ead06` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:39.028738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:39.029108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c1d65-89ae-4e9c-8388-73eefcdf40dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=998108a1ef4b476c8aedaab1aa8ead06)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:39.031799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c1d65-89ae-4e9c-8388-73eefcdf40dc admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:39.099971 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f6ad16c6-591a-4ef8-8201-5560f45e4e12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4019536f1da486cbadd801f3b939585', 'request_id': 'req-f6ad16c6-591a-4ef8-8201-5560f45e4e12'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:39.117747 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f6ad16c6-591a-4ef8-8201-5560f45e4e12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4019536f1da486cbadd801f3b939585', 'request_id': 'req-f6ad16c6-591a-4ef8-8201-5560f45e4e12'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:39.128189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f6ad16c6-591a-4ef8-8201-5560f45e4e12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4019536f1da486cbadd801f3b939585', 'request_id': 'req-f6ad16c6-591a-4ef8-8201-5560f45e4e12'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:39.133453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-da1c1d65-89ae-4e9c-8388-73eefcdf40dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '998108a1ef4b476c8aedaab1aa8ead06'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:39.148578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-da1c1d65-89ae-4e9c-8388-73eefcdf40dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '998108a1ef4b476c8aedaab1aa8ead06'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:39.152557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f6ad16c6-591a-4ef8-8201-5560f45e4e12 admin admin] The token cache is being invalidate because project b4019536f1da486cbadd801f3b939585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:45:39.152940 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f6ad16c6-591a-4ef8-8201-5560f45e4e12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:39.154145 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f6ad16c6-591a-4ef8-8201-5560f45e4e12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:39.162792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1039/2072] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:45:38 2022] DELETE /identity/v3/projects/b4019536f1da486cbadd801f3b939585 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:39.167087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4efdd3-9fbe-4389-9020-bd63e10622cb None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:39.167555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4efdd3-9fbe-4389-9020-bd63e10622cb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:39.168056 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4efdd3-9fbe-4389-9020-bd63e10622cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:39.172444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-da1c1d65-89ae-4e9c-8388-73eefcdf40dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '998108a1ef4b476c8aedaab1aa8ead06'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:39.173217 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-da1c1d65-89ae-4e9c-8388-73eefcdf40dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '998108a1ef4b476c8aedaab1aa8ead06'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:39.174153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-da1c1d65-89ae-4e9c-8388-73eefcdf40dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '998108a1ef4b476c8aedaab1aa8ead06'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:39.221636 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-bb4efdd3-9fbe-4389-9020-bd63e10622cb None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:39.224440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bb4efdd3-9fbe-4389-9020-bd63e10622cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:39.250805 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1034/2073] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:45:38 2022] DELETE /identity/v3/users/998108a1ef4b476c8aedaab1aa8ead06 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:39.252526 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4742778-ab69-4b2c-95bf-0ab5e147ea85 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:39.252921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4742778-ab69-4b2c-95bf-0ab5e147ea85 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:39.253876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4742778-ab69-4b2c-95bf-0ab5e147ea85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:39.290749 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b4742778-ab69-4b2c-95bf-0ab5e147ea85 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:39.296458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b4742778-ab69-4b2c-95bf-0ab5e147ea85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:39.356730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1040/2074] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:39.373519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a691e8-e739-48ec-9e68-b88bc19a9a2e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:39.374246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a691e8-e739-48ec-9e68-b88bc19a9a2e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:39.374641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a691e8-e739-48ec-9e68-b88bc19a9a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:39.416893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e7a691e8-e739-48ec-9e68-b88bc19a9a2e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:39.421142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e7a691e8-e739-48ec-9e68-b88bc19a9a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:39.472180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1035/2075] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:39.491101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4742778-ab69-4b2c-95bf-0ab5e147ea85 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:39.494092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b4742778-ab69-4b2c-95bf-0ab5e147ea85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:39.512580 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:39.587389 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1041/2076] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:45:39.594311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a691e8-e739-48ec-9e68-b88bc19a9a2e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:39.624475 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a691e8-e739-48ec-9e68-b88bc19a9a2e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:39.634154 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e9885-49dd-441b-be35-2f0543b1fad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:39.634894 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e9885-49dd-441b-be35-2f0543b1fad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:39.635237 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e9885-49dd-441b-be35-2f0543b1fad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:39.639215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e9885-49dd-441b-be35-2f0543b1fad2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:39.641563 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e9885-49dd-441b-be35-2f0543b1fad2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:45:39.641899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e9885-49dd-441b-be35-2f0543b1fad2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:39.645972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e9885-49dd-441b-be35-2f0543b1fad2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:39.686308 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1042/2077] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:45:39.686609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4742778-ab69-4b2c-95bf-0ab5e147ea85 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:39.691410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e9885-49dd-441b-be35-2f0543b1fad2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:39.697849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57572487-8292-4b8e-9760-024a7b388a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:39.697849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57572487-8292-4b8e-9760-024a7b388a19 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:39.697849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57572487-8292-4b8e-9760-024a7b388a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:39.697849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57572487-8292-4b8e-9760-024a7b388a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:39.717929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e9885-49dd-441b-be35-2f0543b1fad2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:39.718512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57572487-8292-4b8e-9760-024a7b388a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:39.723660 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57572487-8292-4b8e-9760-024a7b388a19 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:39.742323 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd656d25-deaa-4103-b3a7-96c1a6a6b475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:39.744991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd656d25-deaa-4103-b3a7-96c1a6a6b475 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:39.745910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd656d25-deaa-4103-b3a7-96c1a6a6b475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:39.748064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd656d25-deaa-4103-b3a7-96c1a6a6b475 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:39.750035 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd656d25-deaa-4103-b3a7-96c1a6a6b475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:39.750916 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1036/2078] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:45:39.774644 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd656d25-deaa-4103-b3a7-96c1a6a6b475 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:39.853128 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1043/2079] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:45:39 2022] POST /identity/v3/projects => generated 369 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:39.864407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57572487-8292-4b8e-9760-024a7b388a19 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:39.888910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57572487-8292-4b8e-9760-024a7b388a19 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:40.053962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37e65c-bfa9-401b-9eef-1e084174c329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:40.054886 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37e65c-bfa9-401b-9eef-1e084174c329 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:40.055104 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37e65c-bfa9-401b-9eef-1e084174c329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:40.055316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37e65c-bfa9-401b-9eef-1e084174c329 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:40.056232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37e65c-bfa9-401b-9eef-1e084174c329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:40.058534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37e65c-bfa9-401b-9eef-1e084174c329 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:40.111343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1037/2080] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:39 2022] POST /identity/v3/users => generated 285 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:40.117100 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd656d25-deaa-4103-b3a7-96c1a6a6b475 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:40.137430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd656d25-deaa-4103-b3a7-96c1a6a6b475 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:40.151045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eeb63f-3766-46b8-960c-fdcfc9744445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:40.152152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eeb63f-3766-46b8-960c-fdcfc9744445 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:40.152674 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eeb63f-3766-46b8-960c-fdcfc9744445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:40.153123 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eeb63f-3766-46b8-960c-fdcfc9744445 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:40.154866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eeb63f-3766-46b8-960c-fdcfc9744445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:40.157341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eeb63f-3766-46b8-960c-fdcfc9744445 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:40.161850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:40.185236 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37e65c-bfa9-401b-9eef-1e084174c329 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:40.193712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1044/2081] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:40.198083 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37e65c-bfa9-401b-9eef-1e084174c329 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:40.207168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc771f-e9de-411e-a835-f25c29c0b8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:40.208229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc771f-e9de-411e-a835-f25c29c0b8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:40.208748 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc771f-e9de-411e-a835-f25c29c0b8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:40.209265 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc771f-e9de-411e-a835-f25c29c0b8d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:40.210693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc771f-e9de-411e-a835-f25c29c0b8d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:40.213448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc771f-e9de-411e-a835-f25c29c0b8d0 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:40.233342 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1038/2082] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:45:40.243820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eeb63f-3766-46b8-960c-fdcfc9744445 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:40.266635 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eeb63f-3766-46b8-960c-fdcfc9744445 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:40.275184 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712fbc4-3929-4b12-88b4-2080bfea0e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:40.276101 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712fbc4-3929-4b12-88b4-2080bfea0e9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:40.276463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712fbc4-3929-4b12-88b4-2080bfea0e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:40.276797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712fbc4-3929-4b12-88b4-2080bfea0e9c admin admin] PATH_INFO: `/v3/projects/cf86280b8a364172ab65111a257a8bb6/users/2f336db03c2d48cf8e9be0eaf3578f18/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:40.302015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712fbc4-3929-4b12-88b4-2080bfea0e9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf86280b8a364172ab65111a257a8bb6, user_id=2f336db03c2d48cf8e9be0eaf3578f18, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:40.305398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712fbc4-3929-4b12-88b4-2080bfea0e9c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:40.327874 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1045/2083] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:40 2022] PUT /identity/v3/projects/cf86280b8a364172ab65111a257a8bb6/users/2f336db03c2d48cf8e9be0eaf3578f18/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:40.351253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e32461-1fca-42f7-99bc-869536b2849d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:40.353016 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e32461-1fca-42f7-99bc-869536b2849d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:40.353673 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e32461-1fca-42f7-99bc-869536b2849d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:40.447275 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b7e32461-1fca-42f7-99bc-869536b2849d None None] MFA Rules not processed for user `2f336db03c2d48cf8e9be0eaf3578f18`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:40.451574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b7e32461-1fca-42f7-99bc-869536b2849d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:40.461292 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712fbc4-3929-4b12-88b4-2080bfea0e9c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:40.463937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c712fbc4-3929-4b12-88b4-2080bfea0e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:40.591807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712fbc4-3929-4b12-88b4-2080bfea0e9c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:40.592960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1039/2084] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:40.606381 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e32461-1fca-42f7-99bc-869536b2849d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:40.607001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc2c7a-d074-44df-954a-4f33bd8be7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:40.609130 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc2c7a-d074-44df-954a-4f33bd8be7a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:40.609619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc2c7a-d074-44df-954a-4f33bd8be7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:40.610329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc2c7a-d074-44df-954a-4f33bd8be7a4 admin admin] PATH_INFO: `/v3/projects/790e59abcdf14039869204561c535f75` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:40.610570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b7e32461-1fca-42f7-99bc-869536b2849d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:40.613556 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc2c7a-d074-44df-954a-4f33bd8be7a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=790e59abcdf14039869204561c535f75)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:40.619719 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc2c7a-d074-44df-954a-4f33bd8be7a4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:40.672250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:40.744503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-33fc2c7a-d074-44df-954a-4f33bd8be7a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '790e59abcdf14039869204561c535f75', 'request_id': 'req-33fc2c7a-d074-44df-954a-4f33bd8be7a4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:40.761047 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-33fc2c7a-d074-44df-954a-4f33bd8be7a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '790e59abcdf14039869204561c535f75', 'request_id': 'req-33fc2c7a-d074-44df-954a-4f33bd8be7a4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:40.786219 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-33fc2c7a-d074-44df-954a-4f33bd8be7a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '790e59abcdf14039869204561c535f75', 'request_id': 'req-33fc2c7a-d074-44df-954a-4f33bd8be7a4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:40.819979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-33fc2c7a-d074-44df-954a-4f33bd8be7a4 admin admin] The token cache is being invalidate because project 790e59abcdf14039869204561c535f75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:45:40.822186 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-33fc2c7a-d074-44df-954a-4f33bd8be7a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:40.823636 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-33fc2c7a-d074-44df-954a-4f33bd8be7a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:40.827009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1046/2085] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:45:40 2022] DELETE /identity/v3/projects/790e59abcdf14039869204561c535f75 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:45:40.848955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e32461-1fca-42f7-99bc-869536b2849d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:40.935976 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b019cb9-a8b2-442d-906b-e38cf0bca31f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '300fb5e1c5c546808b935871673e3d0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:40.937276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b019cb9-a8b2-442d-906b-e38cf0bca31f service glance] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:40.937819 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b019cb9-a8b2-442d-906b-e38cf0bca31f service glance] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:40.938249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b019cb9-a8b2-442d-906b-e38cf0bca31f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:40.940998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9b019cb9-a8b2-442d-906b-e38cf0bca31f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:41.068352 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b019cb9-a8b2-442d-906b-e38cf0bca31f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:41.071443 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b019cb9-a8b2-442d-906b-e38cf0bca31f service glance] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:41.120976 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1040/2086] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:41.301968 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef89628-e20c-45b7-bd06-da5a1632e002 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:41.302284 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef89628-e20c-45b7-bd06-da5a1632e002 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:41.302524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef89628-e20c-45b7-bd06-da5a1632e002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:41.329259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9ef89628-e20c-45b7-bd06-da5a1632e002 None None] MFA Rules not processed for user `7cb192800160427e9951a759dd10b228`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:41.330646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9ef89628-e20c-45b7-bd06-da5a1632e002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:41.423857 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1047/2087] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:46.227934 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b019cb9-a8b2-442d-906b-e38cf0bca31f service glance] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:46.229420 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9b019cb9-a8b2-442d-906b-e38cf0bca31f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:46.326858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b019cb9-a8b2-442d-906b-e38cf0bca31f service glance] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:46.331645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977894b3-c297-4080-890c-b82aa986222f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:46.332312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977894b3-c297-4080-890c-b82aa986222f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:46.332594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977894b3-c297-4080-890c-b82aa986222f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:46.332887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977894b3-c297-4080-890c-b82aa986222f admin admin] PATH_INFO: `/v3/users/023bc40a6c274d0c80e0e5a469a1980f` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:46.334099 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977894b3-c297-4080-890c-b82aa986222f admin admin] RBAC: Authorizing `identity:delete_user(user_id=023bc40a6c274d0c80e0e5a469a1980f)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:46.336249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977894b3-c297-4080-890c-b82aa986222f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:46.342439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:46.408256 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-977894b3-c297-4080-890c-b82aa986222f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '023bc40a6c274d0c80e0e5a469a1980f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:46.416517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-977894b3-c297-4080-890c-b82aa986222f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '023bc40a6c274d0c80e0e5a469a1980f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:46.433372 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-977894b3-c297-4080-890c-b82aa986222f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '023bc40a6c274d0c80e0e5a469a1980f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:46.434159 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-977894b3-c297-4080-890c-b82aa986222f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '023bc40a6c274d0c80e0e5a469a1980f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:46.434809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-977894b3-c297-4080-890c-b82aa986222f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '023bc40a6c274d0c80e0e5a469a1980f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:46.456098 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1041/2088] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:45:46 2022] DELETE /identity/v3/users/023bc40a6c274d0c80e0e5a469a1980f => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:46.462093 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b379891-d60e-414f-a776-7b4cb9f3461a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:46.462670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b379891-d60e-414f-a776-7b4cb9f3461a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:46.463173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b379891-d60e-414f-a776-7b4cb9f3461a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:46.488342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9b379891-d60e-414f-a776-7b4cb9f3461a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:46.489689 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9b379891-d60e-414f-a776-7b4cb9f3461a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:46.496555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:46.583702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1048/2089] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:45:46.600703 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977894b3-c297-4080-890c-b82aa986222f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:46.602197 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-977894b3-c297-4080-890c-b82aa986222f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:46.682365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977894b3-c297-4080-890c-b82aa986222f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:46.686121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a333a-6df4-4129-a01c-70f1a6a5bd1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:46.687199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a333a-6df4-4129-a01c-70f1a6a5bd1c service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:46.687199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a333a-6df4-4129-a01c-70f1a6a5bd1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:46.687401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a333a-6df4-4129-a01c-70f1a6a5bd1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:46.698156 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a333a-6df4-4129-a01c-70f1a6a5bd1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:46.699613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a333a-6df4-4129-a01c-70f1a6a5bd1c service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:46.711433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1042/2090] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:47.060935 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b379891-d60e-414f-a776-7b4cb9f3461a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:47.063075 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9b379891-d60e-414f-a776-7b4cb9f3461a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:47.078263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b379891-d60e-414f-a776-7b4cb9f3461a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:47.084671 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6c0de-0fec-46fb-909e-5de09872e67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:47.085556 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6c0de-0fec-46fb-909e-5de09872e67f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:47.085842 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6c0de-0fec-46fb-909e-5de09872e67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:47.086118 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6c0de-0fec-46fb-909e-5de09872e67f admin admin] PATH_INFO: `/v3/projects/06e59c62f01f4f0383b0a9d75b63b7b2` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:47.087355 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6c0de-0fec-46fb-909e-5de09872e67f admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e59c62f01f4f0383b0a9d75b63b7b2)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:47.089614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6c0de-0fec-46fb-909e-5de09872e67f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:47.152262 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6af6c0de-0fec-46fb-909e-5de09872e67f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e59c62f01f4f0383b0a9d75b63b7b2', 'request_id': 'req-6af6c0de-0fec-46fb-909e-5de09872e67f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:47.160892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6af6c0de-0fec-46fb-909e-5de09872e67f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e59c62f01f4f0383b0a9d75b63b7b2', 'request_id': 'req-6af6c0de-0fec-46fb-909e-5de09872e67f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:47.170072 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6af6c0de-0fec-46fb-909e-5de09872e67f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e59c62f01f4f0383b0a9d75b63b7b2', 'request_id': 'req-6af6c0de-0fec-46fb-909e-5de09872e67f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:47.188172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6af6c0de-0fec-46fb-909e-5de09872e67f admin admin] The token cache is being invalidate because project 06e59c62f01f4f0383b0a9d75b63b7b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:45:47.188469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6af6c0de-0fec-46fb-909e-5de09872e67f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:47.189326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6af6c0de-0fec-46fb-909e-5de09872e67f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:47.192360 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1049/2091] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:45:47 2022] DELETE /identity/v3/projects/06e59c62f01f4f0383b0a9d75b63b7b2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:47.197804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cf8de-c02b-4673-b267-ac2f7220034a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:47.198097 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cf8de-c02b-4673-b267-ac2f7220034a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:47.198379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cf8de-c02b-4673-b267-ac2f7220034a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:47.223020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-468cf8de-c02b-4673-b267-ac2f7220034a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:47.224456 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-468cf8de-c02b-4673-b267-ac2f7220034a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:47.308592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1043/2092] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:47.317235 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d750320-8721-4734-901f-8aeae3a36abe None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:47.317432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d750320-8721-4734-901f-8aeae3a36abe None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:47.317747 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d750320-8721-4734-901f-8aeae3a36abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:47.343872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4d750320-8721-4734-901f-8aeae3a36abe None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:47.345114 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4d750320-8721-4734-901f-8aeae3a36abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:47.362164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1050/2093] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:47.368041 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cf8de-c02b-4673-b267-ac2f7220034a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:47.380527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cf8de-c02b-4673-b267-ac2f7220034a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:47.387846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751932f-ab33-4d19-a877-e1eaee948fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:47.388952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751932f-ab33-4d19-a877-e1eaee948fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:47.388952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751932f-ab33-4d19-a877-e1eaee948fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:47.389204 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751932f-ab33-4d19-a877-e1eaee948fc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:47.389800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751932f-ab33-4d19-a877-e1eaee948fc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:45:47.390111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751932f-ab33-4d19-a877-e1eaee948fc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:47.391772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751932f-ab33-4d19-a877-e1eaee948fc7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:47.405577 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1044/2094] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:45:47.411710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d750320-8721-4734-901f-8aeae3a36abe None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:47.422102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d750320-8721-4734-901f-8aeae3a36abe None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:47.429405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb6e86-3226-4ad7-a3a9-0f0671838686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:47.430247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb6e86-3226-4ad7-a3a9-0f0671838686 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:47.430619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb6e86-3226-4ad7-a3a9-0f0671838686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:47.430942 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb6e86-3226-4ad7-a3a9-0f0671838686 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:47.431803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb6e86-3226-4ad7-a3a9-0f0671838686 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:47.434278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb6e86-3226-4ad7-a3a9-0f0671838686 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:47.472933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1051/2095] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:45:47 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:47.477817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751932f-ab33-4d19-a877-e1eaee948fc7 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:47.489182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751932f-ab33-4d19-a877-e1eaee948fc7 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:47.508055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:47.571293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47be20-9124-4be3-a237-1f479dba106a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:47.571939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47be20-9124-4be3-a237-1f479dba106a admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:47.572223 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47be20-9124-4be3-a237-1f479dba106a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:47.572501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47be20-9124-4be3-a237-1f479dba106a admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:47.573182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47be20-9124-4be3-a237-1f479dba106a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:47.576181 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47be20-9124-4be3-a237-1f479dba106a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:47.612896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1045/2096] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:47 2022] POST /identity/v3/users => generated 283 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:47.618932 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb6e86-3226-4ad7-a3a9-0f0671838686 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:47.632008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb6e86-3226-4ad7-a3a9-0f0671838686 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:47.637148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87079e5-da92-4816-99af-227479dcb374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:47.637829 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87079e5-da92-4816-99af-227479dcb374 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:47.638129 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87079e5-da92-4816-99af-227479dcb374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:47.638388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87079e5-da92-4816-99af-227479dcb374 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:47.639110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87079e5-da92-4816-99af-227479dcb374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:47.640714 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87079e5-da92-4816-99af-227479dcb374 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:47.651682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1052/2097] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:47.656079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47be20-9124-4be3-a237-1f479dba106a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:47.666963 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47be20-9124-4be3-a237-1f479dba106a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:47.673226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05dd47-c68f-4784-b437-02df5b36e939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:47.673917 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05dd47-c68f-4784-b437-02df5b36e939 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:47.674199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05dd47-c68f-4784-b437-02df5b36e939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:47.674474 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05dd47-c68f-4784-b437-02df5b36e939 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:47.675761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05dd47-c68f-4784-b437-02df5b36e939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:47.677410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05dd47-c68f-4784-b437-02df5b36e939 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:47.703390 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1046/2098] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:47.705336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87079e5-da92-4816-99af-227479dcb374 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:47.722506 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87079e5-da92-4816-99af-227479dcb374 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:47.728426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f59f0c-c097-4f9f-a77e-e76be9bc061e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:47.729257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f59f0c-c097-4f9f-a77e-e76be9bc061e admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:47.729632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f59f0c-c097-4f9f-a77e-e76be9bc061e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:47.729920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f59f0c-c097-4f9f-a77e-e76be9bc061e admin admin] PATH_INFO: `/v3/projects/e3c038a6bb344d9aa0ca9f1e6530baf9/users/d2bbb8fe4b0f4d8fb0fa8a5869443545/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:47.753968 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f59f0c-c097-4f9f-a77e-e76be9bc061e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3c038a6bb344d9aa0ca9f1e6530baf9, user_id=d2bbb8fe4b0f4d8fb0fa8a5869443545, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:47.756611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f59f0c-c097-4f9f-a77e-e76be9bc061e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:47.771880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1053/2099] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:47 2022] PUT /identity/v3/projects/e3c038a6bb344d9aa0ca9f1e6530baf9/users/d2bbb8fe4b0f4d8fb0fa8a5869443545/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:47.783343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab8333-26d3-45a6-b3b8-17aa58c7a09c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:47.783660 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab8333-26d3-45a6-b3b8-17aa58c7a09c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:47.783903 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab8333-26d3-45a6-b3b8-17aa58c7a09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:47.818825 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d1ab8333-26d3-45a6-b3b8-17aa58c7a09c None None] MFA Rules not processed for user `d2bbb8fe4b0f4d8fb0fa8a5869443545`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:47.820273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d1ab8333-26d3-45a6-b3b8-17aa58c7a09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:47.898795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1047/2100] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:45:47.903940 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f59f0c-c097-4f9f-a77e-e76be9bc061e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:47.946205 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:47.989025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f59f0c-c097-4f9f-a77e-e76be9bc061e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:47.994037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d95abb-2909-4216-b6e8-45f8f5d4ea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:47.994809 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d95abb-2909-4216-b6e8-45f8f5d4ea4e admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:47.995202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d95abb-2909-4216-b6e8-45f8f5d4ea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:47.995485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d95abb-2909-4216-b6e8-45f8f5d4ea4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:47.996347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d95abb-2909-4216-b6e8-45f8f5d4ea4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:47.998166 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d95abb-2909-4216-b6e8-45f8f5d4ea4e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:48.031604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1054/2101] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:45:47 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:48.037803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab8333-26d3-45a6-b3b8-17aa58c7a09c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:48.053224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab8333-26d3-45a6-b3b8-17aa58c7a09c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:48.144605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd98d64-6f9f-4adb-b412-7c88b5be8782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:48.145277 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd98d64-6f9f-4adb-b412-7c88b5be8782 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:48.145727 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd98d64-6f9f-4adb-b412-7c88b5be8782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:48.145838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd98d64-6f9f-4adb-b412-7c88b5be8782 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:48.146564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd98d64-6f9f-4adb-b412-7c88b5be8782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:48.148497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd98d64-6f9f-4adb-b412-7c88b5be8782 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:48.181599 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1048/2102] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:48 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:45:48.183675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d95abb-2909-4216-b6e8-45f8f5d4ea4e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:48.195975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d95abb-2909-4216-b6e8-45f8f5d4ea4e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:48.201291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a99dd-4abe-4b6e-af36-e6bfa59dce00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:48.201995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a99dd-4abe-4b6e-af36-e6bfa59dce00 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:48.202279 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a99dd-4abe-4b6e-af36-e6bfa59dce00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:48.202559 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a99dd-4abe-4b6e-af36-e6bfa59dce00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:48.203312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a99dd-4abe-4b6e-af36-e6bfa59dce00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:48.204924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a99dd-4abe-4b6e-af36-e6bfa59dce00 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:48.216749 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1055/2103] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:45:48.221375 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd98d64-6f9f-4adb-b412-7c88b5be8782 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:48.232426 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd98d64-6f9f-4adb-b412-7c88b5be8782 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:48.239576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa52f5-8c1a-44a0-93ce-18fecfd8e6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:48.240319 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa52f5-8c1a-44a0-93ce-18fecfd8e6b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:48.240605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa52f5-8c1a-44a0-93ce-18fecfd8e6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:48.240877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa52f5-8c1a-44a0-93ce-18fecfd8e6b3 admin admin] PATH_INFO: `/v3/projects/fbdd7d78a1cb4846a4d9e88ac972efe1/users/57a0803049ce4b6c904751666c1c8b35/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:48.253866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:48.259483 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa52f5-8c1a-44a0-93ce-18fecfd8e6b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbdd7d78a1cb4846a4d9e88ac972efe1, user_id=57a0803049ce4b6c904751666c1c8b35, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:48.264109 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa52f5-8c1a-44a0-93ce-18fecfd8e6b3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:48.280521 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1049/2104] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:48 2022] PUT /identity/v3/projects/fbdd7d78a1cb4846a4d9e88ac972efe1/users/57a0803049ce4b6c904751666c1c8b35/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:48.285130 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a99dd-4abe-4b6e-af36-e6bfa59dce00 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:48.301900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a99dd-4abe-4b6e-af36-e6bfa59dce00 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:48.368270 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a19487-a38e-47fe-a773-dfce996628f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:48.368898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a19487-a38e-47fe-a773-dfce996628f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:48.369181 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a19487-a38e-47fe-a773-dfce996628f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:48.369455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a19487-a38e-47fe-a773-dfce996628f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:48.370185 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a19487-a38e-47fe-a773-dfce996628f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:48.371820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a19487-a38e-47fe-a773-dfce996628f7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:48.382690 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1056/2105] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:48.390811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa52f5-8c1a-44a0-93ce-18fecfd8e6b3 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:48.400287 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa52f5-8c1a-44a0-93ce-18fecfd8e6b3 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:48.405320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9a3ad-f58b-485c-bc75-199ad4f03a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:48.406042 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9a3ad-f58b-485c-bc75-199ad4f03a07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:48.406325 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9a3ad-f58b-485c-bc75-199ad4f03a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:48.406598 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9a3ad-f58b-485c-bc75-199ad4f03a07 admin admin] PATH_INFO: `/v3/projects/fbdd7d78a1cb4846a4d9e88ac972efe1/users/57a0803049ce4b6c904751666c1c8b35/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:48.408778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9a3ad-f58b-485c-bc75-199ad4f03a07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbdd7d78a1cb4846a4d9e88ac972efe1, user_id=57a0803049ce4b6c904751666c1c8b35, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:48.411758 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9a3ad-f58b-485c-bc75-199ad4f03a07 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:48.423140 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1050/2106] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:48 2022] PUT /identity/v3/projects/fbdd7d78a1cb4846a4d9e88ac972efe1/users/57a0803049ce4b6c904751666c1c8b35/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:48.435762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a2f4f-439b-4476-b4ae-60ae21d32e49 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:48.436117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a2f4f-439b-4476-b4ae-60ae21d32e49 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:48.436495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a2f4f-439b-4476-b4ae-60ae21d32e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:48.465855 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-254a2f4f-439b-4476-b4ae-60ae21d32e49 None None] MFA Rules not processed for user `57a0803049ce4b6c904751666c1c8b35`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:48.467357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-254a2f4f-439b-4476-b4ae-60ae21d32e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:48.551587 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1057/2107] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:48.572220 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9a3ad-f58b-485c-bc75-199ad4f03a07 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:48.574030 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-dcc9a3ad-f58b-485c-bc75-199ad4f03a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:48.638209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9a3ad-f58b-485c-bc75-199ad4f03a07 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:48.641939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352d9e4-132e-4980-a279-092bf64b3350 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:48.642714 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352d9e4-132e-4980-a279-092bf64b3350 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:48.642993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352d9e4-132e-4980-a279-092bf64b3350 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:48.643268 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352d9e4-132e-4980-a279-092bf64b3350 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:48.659474 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352d9e4-132e-4980-a279-092bf64b3350 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:48.660949 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352d9e4-132e-4980-a279-092bf64b3350 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:48.672432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1051/2108] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:49.174639 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a2f4f-439b-4476-b4ae-60ae21d32e49 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:49.217512 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:49.275216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a2f4f-439b-4476-b4ae-60ae21d32e49 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:49.279767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b14e4-1888-44e7-bb54-f373b1bad810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:49.280923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b14e4-1888-44e7-bb54-f373b1bad810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:49.280923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b14e4-1888-44e7-bb54-f373b1bad810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:49.281216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b14e4-1888-44e7-bb54-f373b1bad810 admin admin] PATH_INFO: `/v3/users/d2bbb8fe4b0f4d8fb0fa8a5869443545` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:49.282299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b14e4-1888-44e7-bb54-f373b1bad810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2bbb8fe4b0f4d8fb0fa8a5869443545)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:49.284278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b14e4-1888-44e7-bb54-f373b1bad810 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:49.351511 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b69b14e4-1888-44e7-bb54-f373b1bad810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2bbb8fe4b0f4d8fb0fa8a5869443545'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:49.370982 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b69b14e4-1888-44e7-bb54-f373b1bad810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2bbb8fe4b0f4d8fb0fa8a5869443545'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:49.372994 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b69b14e4-1888-44e7-bb54-f373b1bad810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2bbb8fe4b0f4d8fb0fa8a5869443545'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:49.385808 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b69b14e4-1888-44e7-bb54-f373b1bad810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2bbb8fe4b0f4d8fb0fa8a5869443545'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:49.396171 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b69b14e4-1888-44e7-bb54-f373b1bad810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2bbb8fe4b0f4d8fb0fa8a5869443545'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:49.405965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1058/2109] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:45:49 2022] DELETE /identity/v3/users/d2bbb8fe4b0f4d8fb0fa8a5869443545 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:49.408423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05295ffc-2790-45b4-8433-cf50730ebcea None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:49.409012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05295ffc-2790-45b4-8433-cf50730ebcea None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:49.409012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05295ffc-2790-45b4-8433-cf50730ebcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:49.432440 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-05295ffc-2790-45b4-8433-cf50730ebcea None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:49.433977 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-05295ffc-2790-45b4-8433-cf50730ebcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:49.510449 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1052/2110] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:45:49.522768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b14e4-1888-44e7-bb54-f373b1bad810 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:49.525606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b69b14e4-1888-44e7-bb54-f373b1bad810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:49.622253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b14e4-1888-44e7-bb54-f373b1bad810 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:49.626789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57ef91-df48-4996-854f-15100214b304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:49.627452 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57ef91-df48-4996-854f-15100214b304 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:49.627830 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57ef91-df48-4996-854f-15100214b304 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:49.628644 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57ef91-df48-4996-854f-15100214b304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:49.640137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57ef91-df48-4996-854f-15100214b304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:49.641287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57ef91-df48-4996-854f-15100214b304 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:49.662653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1059/2111] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:45:50.142088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05295ffc-2790-45b4-8433-cf50730ebcea None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:50.144470 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-05295ffc-2790-45b4-8433-cf50730ebcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:50.158566 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05295ffc-2790-45b4-8433-cf50730ebcea None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:50.163389 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b444b-fef8-4cc0-9f83-c223860dab41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:50.164055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b444b-fef8-4cc0-9f83-c223860dab41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:50.164337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b444b-fef8-4cc0-9f83-c223860dab41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:50.164642 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b444b-fef8-4cc0-9f83-c223860dab41 admin admin] PATH_INFO: `/v3/projects/e3c038a6bb344d9aa0ca9f1e6530baf9` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:50.166191 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b444b-fef8-4cc0-9f83-c223860dab41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3c038a6bb344d9aa0ca9f1e6530baf9)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:50.168158 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b444b-fef8-4cc0-9f83-c223860dab41 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:50.232650 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3a7b444b-fef8-4cc0-9f83-c223860dab41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3c038a6bb344d9aa0ca9f1e6530baf9', 'request_id': 'req-3a7b444b-fef8-4cc0-9f83-c223860dab41'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:50.239467 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3a7b444b-fef8-4cc0-9f83-c223860dab41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3c038a6bb344d9aa0ca9f1e6530baf9', 'request_id': 'req-3a7b444b-fef8-4cc0-9f83-c223860dab41'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:50.248125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3a7b444b-fef8-4cc0-9f83-c223860dab41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3c038a6bb344d9aa0ca9f1e6530baf9', 'request_id': 'req-3a7b444b-fef8-4cc0-9f83-c223860dab41'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:50.264524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3a7b444b-fef8-4cc0-9f83-c223860dab41 admin admin] The token cache is being invalidate because project e3c038a6bb344d9aa0ca9f1e6530baf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:45:50.265348 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3a7b444b-fef8-4cc0-9f83-c223860dab41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:50.265490 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3a7b444b-fef8-4cc0-9f83-c223860dab41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:50.267225 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1053/2112] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:45:50 2022] DELETE /identity/v3/projects/e3c038a6bb344d9aa0ca9f1e6530baf9 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:50.271884 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57ef91-df48-4996-854f-15100214b304 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:50.287287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57ef91-df48-4996-854f-15100214b304 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:50.350965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:50.366769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6833b5a-9c29-4bd9-929b-7b309b4ddb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:50.367474 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6833b5a-9c29-4bd9-929b-7b309b4ddb71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:50.367725 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6833b5a-9c29-4bd9-929b-7b309b4ddb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:50.368001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6833b5a-9c29-4bd9-929b-7b309b4ddb71 admin admin] PATH_INFO: `/v3/users/57a0803049ce4b6c904751666c1c8b35` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:50.369191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6833b5a-9c29-4bd9-929b-7b309b4ddb71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57a0803049ce4b6c904751666c1c8b35)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:50.371153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6833b5a-9c29-4bd9-929b-7b309b4ddb71 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:50.431380 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a6833b5a-9c29-4bd9-929b-7b309b4ddb71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57a0803049ce4b6c904751666c1c8b35'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:50.445909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a6833b5a-9c29-4bd9-929b-7b309b4ddb71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57a0803049ce4b6c904751666c1c8b35'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:50.446675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a6833b5a-9c29-4bd9-929b-7b309b4ddb71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57a0803049ce4b6c904751666c1c8b35'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:50.458011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a6833b5a-9c29-4bd9-929b-7b309b4ddb71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57a0803049ce4b6c904751666c1c8b35'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:50.471051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a6833b5a-9c29-4bd9-929b-7b309b4ddb71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57a0803049ce4b6c904751666c1c8b35'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:50.478516 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1060/2113] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:45:50 2022] DELETE /identity/v3/users/57a0803049ce4b6c904751666c1c8b35 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:45:50.816564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b444b-fef8-4cc0-9f83-c223860dab41 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:50.818225 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3a7b444b-fef8-4cc0-9f83-c223860dab41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:50.833460 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:50.842230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b444b-fef8-4cc0-9f83-c223860dab41 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:50.918026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49326639-efb5-4177-b14e-f6e77b32ef3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:50.918695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49326639-efb5-4177-b14e-f6e77b32ef3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:50.919010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49326639-efb5-4177-b14e-f6e77b32ef3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:50.919324 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49326639-efb5-4177-b14e-f6e77b32ef3a admin admin] PATH_INFO: `/v3/projects/fbdd7d78a1cb4846a4d9e88ac972efe1` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:50.920551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49326639-efb5-4177-b14e-f6e77b32ef3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbdd7d78a1cb4846a4d9e88ac972efe1)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:50.922545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49326639-efb5-4177-b14e-f6e77b32ef3a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:50.992320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-49326639-efb5-4177-b14e-f6e77b32ef3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbdd7d78a1cb4846a4d9e88ac972efe1', 'request_id': 'req-49326639-efb5-4177-b14e-f6e77b32ef3a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:51.003840 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-49326639-efb5-4177-b14e-f6e77b32ef3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbdd7d78a1cb4846a4d9e88ac972efe1', 'request_id': 'req-49326639-efb5-4177-b14e-f6e77b32ef3a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:51.014857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-49326639-efb5-4177-b14e-f6e77b32ef3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbdd7d78a1cb4846a4d9e88ac972efe1', 'request_id': 'req-49326639-efb5-4177-b14e-f6e77b32ef3a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:51.032419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-49326639-efb5-4177-b14e-f6e77b32ef3a admin admin] The token cache is being invalidate because project fbdd7d78a1cb4846a4d9e88ac972efe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:45:51.032711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-49326639-efb5-4177-b14e-f6e77b32ef3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:51.033375 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-49326639-efb5-4177-b14e-f6e77b32ef3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:45:51.035859 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1054/2114] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:45:50 2022] DELETE /identity/v3/projects/fbdd7d78a1cb4846a4d9e88ac972efe1 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:51.041256 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481f91c-ac8e-4c37-84e7-bfe3ceb75976 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:51.042036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481f91c-ac8e-4c37-84e7-bfe3ceb75976 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:51.042036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481f91c-ac8e-4c37-84e7-bfe3ceb75976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:51.069888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b481f91c-ac8e-4c37-84e7-bfe3ceb75976 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:51.071757 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b481f91c-ac8e-4c37-84e7-bfe3ceb75976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:51.155952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1061/2115] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:51.164769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44546b8c-9141-4989-9346-bff7e31144b9 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:51.165037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44546b8c-9141-4989-9346-bff7e31144b9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:51.165366 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44546b8c-9141-4989-9346-bff7e31144b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:51.187915 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-44546b8c-9141-4989-9346-bff7e31144b9 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:51.189153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-44546b8c-9141-4989-9346-bff7e31144b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:51.198118 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1055/2116] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:51.202642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481f91c-ac8e-4c37-84e7-bfe3ceb75976 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:51.216266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481f91c-ac8e-4c37-84e7-bfe3ceb75976 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:51.221760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8a4b5-6483-42fe-bd47-180627bf143c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:51.222365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8a4b5-6483-42fe-bd47-180627bf143c admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:51.222650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8a4b5-6483-42fe-bd47-180627bf143c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:51.222931 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8a4b5-6483-42fe-bd47-180627bf143c admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:51.223656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8a4b5-6483-42fe-bd47-180627bf143c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:45:51.223968 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8a4b5-6483-42fe-bd47-180627bf143c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:51.225653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8a4b5-6483-42fe-bd47-180627bf143c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:51.240768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1062/2117] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:45:51.245616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44546b8c-9141-4989-9346-bff7e31144b9 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:51.258969 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44546b8c-9141-4989-9346-bff7e31144b9 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:51.264178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955620e1-46ad-486b-8a76-ae4d96dec4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:51.264822 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955620e1-46ad-486b-8a76-ae4d96dec4a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:51.265110 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955620e1-46ad-486b-8a76-ae4d96dec4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:51.265385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955620e1-46ad-486b-8a76-ae4d96dec4a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:51.266079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955620e1-46ad-486b-8a76-ae4d96dec4a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:51.267984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955620e1-46ad-486b-8a76-ae4d96dec4a6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:51.300914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1056/2118] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:45:51 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:51.305803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8a4b5-6483-42fe-bd47-180627bf143c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:51.317982 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8a4b5-6483-42fe-bd47-180627bf143c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:51.392245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:51.404109 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56ca11-28ac-4d3d-8e0e-2743a0fad568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:51.404833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56ca11-28ac-4d3d-8e0e-2743a0fad568 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:51.405021 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56ca11-28ac-4d3d-8e0e-2743a0fad568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:51.405021 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56ca11-28ac-4d3d-8e0e-2743a0fad568 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:51.406035 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56ca11-28ac-4d3d-8e0e-2743a0fad568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:51.408057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56ca11-28ac-4d3d-8e0e-2743a0fad568 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:51.439448 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1063/2119] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:51 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:51.441818 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955620e1-46ad-486b-8a76-ae4d96dec4a6 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:51.454810 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955620e1-46ad-486b-8a76-ae4d96dec4a6 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:51.460704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b37d0-43cc-45a2-a2ac-69eb80435b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:51.461348 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b37d0-43cc-45a2-a2ac-69eb80435b34 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:51.461725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b37d0-43cc-45a2-a2ac-69eb80435b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:51.462007 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b37d0-43cc-45a2-a2ac-69eb80435b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:51.462712 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b37d0-43cc-45a2-a2ac-69eb80435b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:51.464384 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b37d0-43cc-45a2-a2ac-69eb80435b34 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:51.475901 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1057/2120] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:51.480079 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56ca11-28ac-4d3d-8e0e-2743a0fad568 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:51.495531 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56ca11-28ac-4d3d-8e0e-2743a0fad568 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:51.503737 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7dc75-a26b-42e6-a0b7-7837ca567169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:51.504921 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7dc75-a26b-42e6-a0b7-7837ca567169 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:51.505789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7dc75-a26b-42e6-a0b7-7837ca567169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:51.505921 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7dc75-a26b-42e6-a0b7-7837ca567169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:51.507187 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7dc75-a26b-42e6-a0b7-7837ca567169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:51.510145 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7dc75-a26b-42e6-a0b7-7837ca567169 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:51.522974 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1064/2121] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:51.528054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b37d0-43cc-45a2-a2ac-69eb80435b34 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:51.539114 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b37d0-43cc-45a2-a2ac-69eb80435b34 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:51.547689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8c614-c4fd-4f1f-b76d-2f329f1e3741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:51.548399 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8c614-c4fd-4f1f-b76d-2f329f1e3741 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:51.548685 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8c614-c4fd-4f1f-b76d-2f329f1e3741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:51.548963 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8c614-c4fd-4f1f-b76d-2f329f1e3741 admin admin] PATH_INFO: `/v3/projects/e2386f2bd47e4163b879b012a0b90eb1/users/79f9f76910434a40a58f445ed8ad33e6/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:51.566504 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8c614-c4fd-4f1f-b76d-2f329f1e3741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2386f2bd47e4163b879b012a0b90eb1, user_id=79f9f76910434a40a58f445ed8ad33e6, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:51.569210 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8c614-c4fd-4f1f-b76d-2f329f1e3741 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:51.583451 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1058/2122] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:51 2022] PUT /identity/v3/projects/e2386f2bd47e4163b879b012a0b90eb1/users/79f9f76910434a40a58f445ed8ad33e6/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:51.594690 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cc661-2efd-4749-89c1-3271757a21af None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:51.594965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cc661-2efd-4749-89c1-3271757a21af None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:51.595482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cc661-2efd-4749-89c1-3271757a21af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:51.627217 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-fb1cc661-2efd-4749-89c1-3271757a21af None None] MFA Rules not processed for user `79f9f76910434a40a58f445ed8ad33e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:51.628482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fb1cc661-2efd-4749-89c1-3271757a21af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:51.722800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1065/2123] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:51.728173 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8c614-c4fd-4f1f-b76d-2f329f1e3741 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:51.767975 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:51.819336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8c614-c4fd-4f1f-b76d-2f329f1e3741 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:51.824595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea241da-247f-46ba-a536-d88925f90bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:51.825177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea241da-247f-46ba-a536-d88925f90bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:51.825603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea241da-247f-46ba-a536-d88925f90bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:51.825869 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea241da-247f-46ba-a536-d88925f90bf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:51.826535 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea241da-247f-46ba-a536-d88925f90bf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:51.828687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea241da-247f-46ba-a536-d88925f90bf1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:51.869629 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1059/2124] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:45:51 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:45:51.872429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cc661-2efd-4749-89c1-3271757a21af None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:51.884816 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cc661-2efd-4749-89c1-3271757a21af None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:51.971060 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8cb4f-e336-4f66-8f76-cfb4adfd0c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:51.971841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8cb4f-e336-4f66-8f76-cfb4adfd0c91 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:51.972614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8cb4f-e336-4f66-8f76-cfb4adfd0c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:51.972908 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8cb4f-e336-4f66-8f76-cfb4adfd0c91 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:51.973627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8cb4f-e336-4f66-8f76-cfb4adfd0c91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:51.975590 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8cb4f-e336-4f66-8f76-cfb4adfd0c91 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:52.001709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1066/2125] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:51 2022] POST /identity/v3/users => generated 294 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:52.005953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea241da-247f-46ba-a536-d88925f90bf1 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:52.016732 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea241da-247f-46ba-a536-d88925f90bf1 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:52.022543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08274131-466a-498d-83e2-f1032159dd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:52.023403 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08274131-466a-498d-83e2-f1032159dd37 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:52.023850 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08274131-466a-498d-83e2-f1032159dd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:52.024212 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08274131-466a-498d-83e2-f1032159dd37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:52.025141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08274131-466a-498d-83e2-f1032159dd37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:52.026866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08274131-466a-498d-83e2-f1032159dd37 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:52.039765 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1060/2126] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:52.043810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8cb4f-e336-4f66-8f76-cfb4adfd0c91 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:52.054124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8cb4f-e336-4f66-8f76-cfb4adfd0c91 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:52.058953 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ae710-de5b-4ec9-8139-f2d3ccfdb07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:52.059577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ae710-de5b-4ec9-8139-f2d3ccfdb07b admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:52.059854 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ae710-de5b-4ec9-8139-f2d3ccfdb07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:52.060126 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ae710-de5b-4ec9-8139-f2d3ccfdb07b admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:52.060866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ae710-de5b-4ec9-8139-f2d3ccfdb07b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:52.062499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ae710-de5b-4ec9-8139-f2d3ccfdb07b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:52.073189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1067/2127] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:45:52.077457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08274131-466a-498d-83e2-f1032159dd37 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:52.088682 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08274131-466a-498d-83e2-f1032159dd37 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:52.093312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791410b6-8037-4ee8-96d2-baa5e28fd1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:52.094008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791410b6-8037-4ee8-96d2-baa5e28fd1d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:52.094295 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791410b6-8037-4ee8-96d2-baa5e28fd1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:52.094560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791410b6-8037-4ee8-96d2-baa5e28fd1d0 admin admin] PATH_INFO: `/v3/projects/4a0d678ab9d84119a54aeb366c01abaf/users/8dbadd3f5ac946f6921c08549115eb1f/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:52.111230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791410b6-8037-4ee8-96d2-baa5e28fd1d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a0d678ab9d84119a54aeb366c01abaf, user_id=8dbadd3f5ac946f6921c08549115eb1f, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:52.114210 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791410b6-8037-4ee8-96d2-baa5e28fd1d0 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:52.133340 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1061/2128] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:52 2022] PUT /identity/v3/projects/4a0d678ab9d84119a54aeb366c01abaf/users/8dbadd3f5ac946f6921c08549115eb1f/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:52.146216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a6cd0-593a-4910-9d68-6544afafb1bb None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:52.146587 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a6cd0-593a-4910-9d68-6544afafb1bb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:52.146942 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a6cd0-593a-4910-9d68-6544afafb1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:52.177359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f58a6cd0-593a-4910-9d68-6544afafb1bb None None] MFA Rules not processed for user `8dbadd3f5ac946f6921c08549115eb1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:52.178666 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f58a6cd0-593a-4910-9d68-6544afafb1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:52.210316 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:52.255406 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1068/2129] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:52.259708 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791410b6-8037-4ee8-96d2-baa5e28fd1d0 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:52.329195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791410b6-8037-4ee8-96d2-baa5e28fd1d0 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:52.334482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5520eb-fd3d-4118-9c0a-ee4a3d5fc7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:52.335135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5520eb-fd3d-4118-9c0a-ee4a3d5fc7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:52.335446 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5520eb-fd3d-4118-9c0a-ee4a3d5fc7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:52.335740 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5520eb-fd3d-4118-9c0a-ee4a3d5fc7a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:52.336426 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5520eb-fd3d-4118-9c0a-ee4a3d5fc7a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:52.338442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5520eb-fd3d-4118-9c0a-ee4a3d5fc7a2 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:52.368837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1062/2130] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:45:52 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:45:52.373365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a6cd0-593a-4910-9d68-6544afafb1bb None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:52.385265 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a6cd0-593a-4910-9d68-6544afafb1bb None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:52.447550 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba4fac-2afe-4195-be25-797fd62c06dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:52.448196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba4fac-2afe-4195-be25-797fd62c06dc admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:52.448480 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba4fac-2afe-4195-be25-797fd62c06dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:52.448759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba4fac-2afe-4195-be25-797fd62c06dc admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:52.449453 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba4fac-2afe-4195-be25-797fd62c06dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:52.451430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba4fac-2afe-4195-be25-797fd62c06dc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:52.477953 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1069/2131] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:45:52 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:45:52.479317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5520eb-fd3d-4118-9c0a-ee4a3d5fc7a2 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:52.490624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5520eb-fd3d-4118-9c0a-ee4a3d5fc7a2 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:52.495292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8a9ad-3000-4031-947d-bb47234194ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:52.495915 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8a9ad-3000-4031-947d-bb47234194ad admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:52.496195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8a9ad-3000-4031-947d-bb47234194ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:52.496565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8a9ad-3000-4031-947d-bb47234194ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:52.497168 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8a9ad-3000-4031-947d-bb47234194ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:52.498800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8a9ad-3000-4031-947d-bb47234194ad admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:52.510226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1063/2132] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:45:52.514564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba4fac-2afe-4195-be25-797fd62c06dc admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:52.525846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba4fac-2afe-4195-be25-797fd62c06dc admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:52.531191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e267a5f-7bb2-4f04-afb2-3984f186fc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:52.531884 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e267a5f-7bb2-4f04-afb2-3984f186fc02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:52.532191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e267a5f-7bb2-4f04-afb2-3984f186fc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:52.532467 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e267a5f-7bb2-4f04-afb2-3984f186fc02 admin admin] PATH_INFO: `/v3/projects/d1b6131bdb0a48b7975ca152c1b5e9aa/users/530d05b380eb41828c355392e37d5780/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:52.549170 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e267a5f-7bb2-4f04-afb2-3984f186fc02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1b6131bdb0a48b7975ca152c1b5e9aa, user_id=530d05b380eb41828c355392e37d5780, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:52.552533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e267a5f-7bb2-4f04-afb2-3984f186fc02 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:52.569328 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1070/2133] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:52 2022] PUT /identity/v3/projects/d1b6131bdb0a48b7975ca152c1b5e9aa/users/530d05b380eb41828c355392e37d5780/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:52.573644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8a9ad-3000-4031-947d-bb47234194ad admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:52.585422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8a9ad-3000-4031-947d-bb47234194ad admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:52.636642 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:52.650354 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d91af2-bbe6-462b-a964-693ce0d84945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:52.651292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d91af2-bbe6-462b-a964-693ce0d84945 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:52.651750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d91af2-bbe6-462b-a964-693ce0d84945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:52.652181 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d91af2-bbe6-462b-a964-693ce0d84945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:52.653066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d91af2-bbe6-462b-a964-693ce0d84945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:52.655155 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d91af2-bbe6-462b-a964-693ce0d84945 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:52.666601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1064/2134] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:45:52.670979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e267a5f-7bb2-4f04-afb2-3984f186fc02 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:52.681907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e267a5f-7bb2-4f04-afb2-3984f186fc02 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:52.687339 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3326c9-f448-42b1-94ad-7e5f8cd52a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:52.688023 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3326c9-f448-42b1-94ad-7e5f8cd52a6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:52.688306 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3326c9-f448-42b1-94ad-7e5f8cd52a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:52.688675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3326c9-f448-42b1-94ad-7e5f8cd52a6c admin admin] PATH_INFO: `/v3/projects/d1b6131bdb0a48b7975ca152c1b5e9aa/users/530d05b380eb41828c355392e37d5780/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:52.691185 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3326c9-f448-42b1-94ad-7e5f8cd52a6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1b6131bdb0a48b7975ca152c1b5e9aa, user_id=530d05b380eb41828c355392e37d5780, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:52.693864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3326c9-f448-42b1-94ad-7e5f8cd52a6c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:52.704379 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1071/2135] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:45:52 2022] PUT /identity/v3/projects/d1b6131bdb0a48b7975ca152c1b5e9aa/users/530d05b380eb41828c355392e37d5780/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:45:52.715629 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f064b-4e19-4e19-851e-fe7904e1c834 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:52.715955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f064b-4e19-4e19-851e-fe7904e1c834 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:52.716268 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f064b-4e19-4e19-851e-fe7904e1c834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:52.745446 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-311f064b-4e19-4e19-851e-fe7904e1c834 None None] MFA Rules not processed for user `530d05b380eb41828c355392e37d5780`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:52.747081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-311f064b-4e19-4e19-851e-fe7904e1c834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:52.821947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1065/2136] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:45:52.834263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3326c9-f448-42b1-94ad-7e5f8cd52a6c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:52.836313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bd3326c9-f448-42b1-94ad-7e5f8cd52a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:52.905067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3326c9-f448-42b1-94ad-7e5f8cd52a6c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:52.909033 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de7936-9d2d-4c9f-a513-2c0cd74a9c21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:52.910313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de7936-9d2d-4c9f-a513-2c0cd74a9c21 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:52.910600 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de7936-9d2d-4c9f-a513-2c0cd74a9c21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:52.910910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de7936-9d2d-4c9f-a513-2c0cd74a9c21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:52.920937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de7936-9d2d-4c9f-a513-2c0cd74a9c21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:52.922424 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de7936-9d2d-4c9f-a513-2c0cd74a9c21 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:52.933347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1072/2137] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:53.563690 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f064b-4e19-4e19-851e-fe7904e1c834 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:53.576185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f064b-4e19-4e19-851e-fe7904e1c834 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:53.581584 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaae0a7-7e9b-43e2-9025-c28b180cd7f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:53.582234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaae0a7-7e9b-43e2-9025-c28b180cd7f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:53.582519 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaae0a7-7e9b-43e2-9025-c28b180cd7f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:53.582798 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaae0a7-7e9b-43e2-9025-c28b180cd7f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:53.659201 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaae0a7-7e9b-43e2-9025-c28b180cd7f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:53.660761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaae0a7-7e9b-43e2-9025-c28b180cd7f5 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:53.672738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1066/2138] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:45:58.039643 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de7936-9d2d-4c9f-a513-2c0cd74a9c21 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:45:58.048707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de7936-9d2d-4c9f-a513-2c0cd74a9c21 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:45:58.053005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5742c-a61b-4496-972c-fd6da264e718 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:45:58.053750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5742c-a61b-4496-972c-fd6da264e718 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:58.054028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5742c-a61b-4496-972c-fd6da264e718 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:58.054315 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5742c-a61b-4496-972c-fd6da264e718 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:58.083241 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:45:58.129444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5742c-a61b-4496-972c-fd6da264e718 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:45:58.130928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5742c-a61b-4496-972c-fd6da264e718 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:45:58.140198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1073/2139] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:45:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:45:58.782176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaf043-e56f-4d3e-a89a-aad758bb53e5 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:45:58.782654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaf043-e56f-4d3e-a89a-aad758bb53e5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:45:58.782734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaf043-e56f-4d3e-a89a-aad758bb53e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:45:58.813700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4afaf043-e56f-4d3e-a89a-aad758bb53e5 None None] MFA Rules not processed for user `956aff831fa04d29ba5cffb88e73f4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:45:58.815118 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4afaf043-e56f-4d3e-a89a-aad758bb53e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:45:58.889361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1067/2140] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:45:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:00.427173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5742c-a61b-4496-972c-fd6da264e718 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:00.499210 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5742c-a61b-4496-972c-fd6da264e718 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:00.504344 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f147e-d2f6-4dda-9b60-7f72d6fea784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:00.505005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f147e-d2f6-4dda-9b60-7f72d6fea784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:00.505292 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f147e-d2f6-4dda-9b60-7f72d6fea784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:00.505603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f147e-d2f6-4dda-9b60-7f72d6fea784 admin admin] PATH_INFO: `/v3/users/79f9f76910434a40a58f445ed8ad33e6` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:00.506818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f147e-d2f6-4dda-9b60-7f72d6fea784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79f9f76910434a40a58f445ed8ad33e6)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:00.508741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f147e-d2f6-4dda-9b60-7f72d6fea784 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:00.565023 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-de7f147e-d2f6-4dda-9b60-7f72d6fea784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79f9f76910434a40a58f445ed8ad33e6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:00.577434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-de7f147e-d2f6-4dda-9b60-7f72d6fea784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79f9f76910434a40a58f445ed8ad33e6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:00.578175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-de7f147e-d2f6-4dda-9b60-7f72d6fea784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79f9f76910434a40a58f445ed8ad33e6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:00.586667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-de7f147e-d2f6-4dda-9b60-7f72d6fea784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79f9f76910434a40a58f445ed8ad33e6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:00.593681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-de7f147e-d2f6-4dda-9b60-7f72d6fea784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79f9f76910434a40a58f445ed8ad33e6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:00.603474 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1074/2141] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:46:00 2022] DELETE /identity/v3/users/79f9f76910434a40a58f445ed8ad33e6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:00.608472 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d643db4-0bc8-405c-a328-252f12c5c3ea None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:00.609387 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d643db4-0bc8-405c-a328-252f12c5c3ea None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:00.609387 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d643db4-0bc8-405c-a328-252f12c5c3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:00.627622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4d643db4-0bc8-405c-a328-252f12c5c3ea None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:00.629392 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4d643db4-0bc8-405c-a328-252f12c5c3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:00.641047 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:00.692519 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1068/2142] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:00.702469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f147e-d2f6-4dda-9b60-7f72d6fea784 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:00.703922 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-de7f147e-d2f6-4dda-9b60-7f72d6fea784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:00.753892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:00.766872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f147e-d2f6-4dda-9b60-7f72d6fea784 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:00.771112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9be05-0785-412a-a18f-6743972a48a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:00.771735 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9be05-0785-412a-a18f-6743972a48a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:00.772012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9be05-0785-412a-a18f-6743972a48a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:00.772285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9be05-0785-412a-a18f-6743972a48a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:00.781677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9be05-0785-412a-a18f-6743972a48a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:00.783111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9be05-0785-412a-a18f-6743972a48a5 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:00.795199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1075/2143] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:46:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:46:01.177050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d643db4-0bc8-405c-a328-252f12c5c3ea None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:01.178608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4d643db4-0bc8-405c-a328-252f12c5c3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:01.193729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d643db4-0bc8-405c-a328-252f12c5c3ea None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:01.198788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5733a-4812-4185-8849-d541423f2995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:01.199548 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5733a-4812-4185-8849-d541423f2995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:01.199895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5733a-4812-4185-8849-d541423f2995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:01.200236 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5733a-4812-4185-8849-d541423f2995 admin admin] PATH_INFO: `/v3/projects/e2386f2bd47e4163b879b012a0b90eb1` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:01.201623 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5733a-4812-4185-8849-d541423f2995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2386f2bd47e4163b879b012a0b90eb1)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:01.203748 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5733a-4812-4185-8849-d541423f2995 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:01.262660 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c4e5733a-4812-4185-8849-d541423f2995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2386f2bd47e4163b879b012a0b90eb1', 'request_id': 'req-c4e5733a-4812-4185-8849-d541423f2995'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:01.284741 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c4e5733a-4812-4185-8849-d541423f2995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2386f2bd47e4163b879b012a0b90eb1', 'request_id': 'req-c4e5733a-4812-4185-8849-d541423f2995'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:01.298338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c4e5733a-4812-4185-8849-d541423f2995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2386f2bd47e4163b879b012a0b90eb1', 'request_id': 'req-c4e5733a-4812-4185-8849-d541423f2995'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:01.330715 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c4e5733a-4812-4185-8849-d541423f2995 admin admin] The token cache is being invalidate because project e2386f2bd47e4163b879b012a0b90eb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:46:01.332041 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c4e5733a-4812-4185-8849-d541423f2995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:01.333341 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c4e5733a-4812-4185-8849-d541423f2995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:01.335907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1069/2144] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:46:01 2022] DELETE /identity/v3/projects/e2386f2bd47e4163b879b012a0b90eb1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:01.339769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9be05-0785-412a-a18f-6743972a48a5 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:01.351911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9be05-0785-412a-a18f-6743972a48a5 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:01.414177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142f3d7-082b-4921-b4e8-59584014fad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:01.414976 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142f3d7-082b-4921-b4e8-59584014fad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:01.415312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142f3d7-082b-4921-b4e8-59584014fad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:01.415774 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142f3d7-082b-4921-b4e8-59584014fad0 admin admin] PATH_INFO: `/v3/users/8dbadd3f5ac946f6921c08549115eb1f` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:01.417640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7142f3d7-082b-4921-b4e8-59584014fad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dbadd3f5ac946f6921c08549115eb1f)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:01.419481 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7142f3d7-082b-4921-b4e8-59584014fad0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:01.477505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7142f3d7-082b-4921-b4e8-59584014fad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dbadd3f5ac946f6921c08549115eb1f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:01.495285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7142f3d7-082b-4921-b4e8-59584014fad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dbadd3f5ac946f6921c08549115eb1f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:01.496031 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7142f3d7-082b-4921-b4e8-59584014fad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dbadd3f5ac946f6921c08549115eb1f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:01.505290 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7142f3d7-082b-4921-b4e8-59584014fad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dbadd3f5ac946f6921c08549115eb1f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:01.522919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7142f3d7-082b-4921-b4e8-59584014fad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dbadd3f5ac946f6921c08549115eb1f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:01.537542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1076/2145] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:46:01 2022] DELETE /identity/v3/users/8dbadd3f5ac946f6921c08549115eb1f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:01.892091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5733a-4812-4185-8849-d541423f2995 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:01.893644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c4e5733a-4812-4185-8849-d541423f2995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:01.918846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5733a-4812-4185-8849-d541423f2995 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:02.009422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ae153-503e-4665-ab7e-b6854409681c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:02.010439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ae153-503e-4665-ab7e-b6854409681c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:02.010902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ae153-503e-4665-ab7e-b6854409681c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:02.011353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ae153-503e-4665-ab7e-b6854409681c admin admin] PATH_INFO: `/v3/projects/4a0d678ab9d84119a54aeb366c01abaf` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:02.012615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ae153-503e-4665-ab7e-b6854409681c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a0d678ab9d84119a54aeb366c01abaf)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:02.014652 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ae153-503e-4665-ab7e-b6854409681c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:02.062721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:02.066899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4f8ae153-503e-4665-ab7e-b6854409681c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a0d678ab9d84119a54aeb366c01abaf', 'request_id': 'req-4f8ae153-503e-4665-ab7e-b6854409681c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:02.074537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4f8ae153-503e-4665-ab7e-b6854409681c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a0d678ab9d84119a54aeb366c01abaf', 'request_id': 'req-4f8ae153-503e-4665-ab7e-b6854409681c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:02.082715 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4f8ae153-503e-4665-ab7e-b6854409681c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a0d678ab9d84119a54aeb366c01abaf', 'request_id': 'req-4f8ae153-503e-4665-ab7e-b6854409681c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:02.094219 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4f8ae153-503e-4665-ab7e-b6854409681c admin admin] The token cache is being invalidate because project 4a0d678ab9d84119a54aeb366c01abaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:46:02.094516 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4f8ae153-503e-4665-ab7e-b6854409681c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:02.095242 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4f8ae153-503e-4665-ab7e-b6854409681c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:02.097915 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1070/2146] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:46:01 2022] DELETE /identity/v3/projects/4a0d678ab9d84119a54aeb366c01abaf => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:02.103079 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142f3d7-082b-4921-b4e8-59584014fad0 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:02.104626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7142f3d7-082b-4921-b4e8-59584014fad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:02.124873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142f3d7-082b-4921-b4e8-59584014fad0 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:02.201191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0deec9-6e26-4fea-aa6e-00327788de46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:02.201895 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0deec9-6e26-4fea-aa6e-00327788de46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:02.202224 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0deec9-6e26-4fea-aa6e-00327788de46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:02.202508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0deec9-6e26-4fea-aa6e-00327788de46 admin admin] PATH_INFO: `/v3/users/530d05b380eb41828c355392e37d5780` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:02.203683 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0deec9-6e26-4fea-aa6e-00327788de46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=530d05b380eb41828c355392e37d5780)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:02.205684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0deec9-6e26-4fea-aa6e-00327788de46 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:02.211186 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:02.262134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5b0deec9-6e26-4fea-aa6e-00327788de46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '530d05b380eb41828c355392e37d5780'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:02.277800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5b0deec9-6e26-4fea-aa6e-00327788de46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '530d05b380eb41828c355392e37d5780'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:02.278709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5b0deec9-6e26-4fea-aa6e-00327788de46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '530d05b380eb41828c355392e37d5780'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:02.298720 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5b0deec9-6e26-4fea-aa6e-00327788de46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '530d05b380eb41828c355392e37d5780'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:02.311955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5b0deec9-6e26-4fea-aa6e-00327788de46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '530d05b380eb41828c355392e37d5780'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:02.319522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1077/2147] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:46:02 2022] DELETE /identity/v3/users/530d05b380eb41828c355392e37d5780 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:02.640399 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ae153-503e-4665-ab7e-b6854409681c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:02.647307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4f8ae153-503e-4665-ab7e-b6854409681c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:02.665033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ae153-503e-4665-ab7e-b6854409681c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:02.743151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb8203-3661-4b5b-994a-9f6c9a7028b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:02.743950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb8203-3661-4b5b-994a-9f6c9a7028b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:02.744517 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb8203-3661-4b5b-994a-9f6c9a7028b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:02.744661 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb8203-3661-4b5b-994a-9f6c9a7028b9 admin admin] PATH_INFO: `/v3/projects/d1b6131bdb0a48b7975ca152c1b5e9aa` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:02.746251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb8203-3661-4b5b-994a-9f6c9a7028b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1b6131bdb0a48b7975ca152c1b5e9aa)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:02.748248 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb8203-3661-4b5b-994a-9f6c9a7028b9 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:02.799623 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-92fb8203-3661-4b5b-994a-9f6c9a7028b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1b6131bdb0a48b7975ca152c1b5e9aa', 'request_id': 'req-92fb8203-3661-4b5b-994a-9f6c9a7028b9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:02.806490 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-92fb8203-3661-4b5b-994a-9f6c9a7028b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1b6131bdb0a48b7975ca152c1b5e9aa', 'request_id': 'req-92fb8203-3661-4b5b-994a-9f6c9a7028b9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:02.814219 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-92fb8203-3661-4b5b-994a-9f6c9a7028b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1b6131bdb0a48b7975ca152c1b5e9aa', 'request_id': 'req-92fb8203-3661-4b5b-994a-9f6c9a7028b9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:02.825234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-92fb8203-3661-4b5b-994a-9f6c9a7028b9 admin admin] The token cache is being invalidate because project d1b6131bdb0a48b7975ca152c1b5e9aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:46:02.825552 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-92fb8203-3661-4b5b-994a-9f6c9a7028b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:02.826190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-92fb8203-3661-4b5b-994a-9f6c9a7028b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:02.830007 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1071/2148] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:46:02 2022] DELETE /identity/v3/projects/d1b6131bdb0a48b7975ca152c1b5e9aa => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:02.833741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acc469-f42e-417a-8920-3d43714fc5e4 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:02.834270 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acc469-f42e-417a-8920-3d43714fc5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:02.834366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acc469-f42e-417a-8920-3d43714fc5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:02.855630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-13acc469-f42e-417a-8920-3d43714fc5e4 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:02.856855 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-13acc469-f42e-417a-8920-3d43714fc5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:02.916272 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1078/2149] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:02.926188 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e50d56-a15a-46ea-bab4-31caa1f623c1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:02.926714 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e50d56-a15a-46ea-bab4-31caa1f623c1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:02.926834 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e50d56-a15a-46ea-bab4-31caa1f623c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:02.953783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-05e50d56-a15a-46ea-bab4-31caa1f623c1 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:02.955301 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-05e50d56-a15a-46ea-bab4-31caa1f623c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:02.967564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1072/2150] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:02.972716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acc469-f42e-417a-8920-3d43714fc5e4 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:02.983889 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acc469-f42e-417a-8920-3d43714fc5e4 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:02.990662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3937e2-8890-4bf1-ae2b-bd481cb2ab76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:02.991448 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3937e2-8890-4bf1-ae2b-bd481cb2ab76 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:02.991882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3937e2-8890-4bf1-ae2b-bd481cb2ab76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:02.992249 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3937e2-8890-4bf1-ae2b-bd481cb2ab76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:02.993416 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3937e2-8890-4bf1-ae2b-bd481cb2ab76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:46:02.993416 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3937e2-8890-4bf1-ae2b-bd481cb2ab76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:02.995459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3937e2-8890-4bf1-ae2b-bd481cb2ab76 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:03.010250 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1079/2151] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:46:03.015990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e50d56-a15a-46ea-bab4-31caa1f623c1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:03.039564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e50d56-a15a-46ea-bab4-31caa1f623c1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:03.045264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c2213-9d75-463f-aab7-0564a345c121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:03.045921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c2213-9d75-463f-aab7-0564a345c121 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:03.046205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c2213-9d75-463f-aab7-0564a345c121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:03.046481 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c2213-9d75-463f-aab7-0564a345c121 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:03.047149 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c2213-9d75-463f-aab7-0564a345c121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:03.049087 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c2213-9d75-463f-aab7-0564a345c121 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:03.077984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1073/2152] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:46:03 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:46:03.082886 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3937e2-8890-4bf1-ae2b-bd481cb2ab76 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:03.093933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3937e2-8890-4bf1-ae2b-bd481cb2ab76 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:03.151032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a4eec-12f4-4877-8bc3-1817ce3900be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:03.151707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a4eec-12f4-4877-8bc3-1817ce3900be admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:03.151991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a4eec-12f4-4877-8bc3-1817ce3900be admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:03.152269 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a4eec-12f4-4877-8bc3-1817ce3900be admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:03.152943 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a4eec-12f4-4877-8bc3-1817ce3900be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:03.155020 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a4eec-12f4-4877-8bc3-1817ce3900be admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:03.175455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:03.185130 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1080/2153] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:46:03 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:46:03.186734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c2213-9d75-463f-aab7-0564a345c121 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:03.198275 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c2213-9d75-463f-aab7-0564a345c121 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:03.205499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd56e4-98d0-4db7-b914-97763d987273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:03.206896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd56e4-98d0-4db7-b914-97763d987273 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:03.207184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd56e4-98d0-4db7-b914-97763d987273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:03.207453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd56e4-98d0-4db7-b914-97763d987273 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:03.208151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd56e4-98d0-4db7-b914-97763d987273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:03.209752 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd56e4-98d0-4db7-b914-97763d987273 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:03.220053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1074/2154] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:46:03.224304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a4eec-12f4-4877-8bc3-1817ce3900be admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:03.234995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a4eec-12f4-4877-8bc3-1817ce3900be admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:03.240337 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0097e-8c13-4be6-9d51-a488e434b141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:03.240958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0097e-8c13-4be6-9d51-a488e434b141 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:03.241241 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0097e-8c13-4be6-9d51-a488e434b141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:03.241555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0097e-8c13-4be6-9d51-a488e434b141 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:03.242255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0097e-8c13-4be6-9d51-a488e434b141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:03.243846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0097e-8c13-4be6-9d51-a488e434b141 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:03.255163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1081/2155] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:46:03.259470 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd56e4-98d0-4db7-b914-97763d987273 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:03.269718 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd56e4-98d0-4db7-b914-97763d987273 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:03.274398 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36761a-7b24-4d8c-8c2c-b9f8f3d24386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:03.275068 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36761a-7b24-4d8c-8c2c-b9f8f3d24386 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:03.275351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36761a-7b24-4d8c-8c2c-b9f8f3d24386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:03.275630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36761a-7b24-4d8c-8c2c-b9f8f3d24386 admin admin] PATH_INFO: `/v3/projects/2332cd679d6c429d9ddcb39948bd4914/users/7175291123eb4eceaa3b17aec9cf3e3e/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:03.290962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36761a-7b24-4d8c-8c2c-b9f8f3d24386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2332cd679d6c429d9ddcb39948bd4914, user_id=7175291123eb4eceaa3b17aec9cf3e3e, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:03.293748 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36761a-7b24-4d8c-8c2c-b9f8f3d24386 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:03.302155 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:03.308870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1075/2156] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:46:03 2022] PUT /identity/v3/projects/2332cd679d6c429d9ddcb39948bd4914/users/7175291123eb4eceaa3b17aec9cf3e3e/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:03.320736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fd2c8-ef3b-4ea8-b589-32474f79b07f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:03.321021 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fd2c8-ef3b-4ea8-b589-32474f79b07f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:03.321296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fd2c8-ef3b-4ea8-b589-32474f79b07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:03.349395 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-fb3fd2c8-ef3b-4ea8-b589-32474f79b07f None None] MFA Rules not processed for user `7175291123eb4eceaa3b17aec9cf3e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:03.350576 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fb3fd2c8-ef3b-4ea8-b589-32474f79b07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:03.422058 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1082/2157] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:03.427176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36761a-7b24-4d8c-8c2c-b9f8f3d24386 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:03.522312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36761a-7b24-4d8c-8c2c-b9f8f3d24386 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:03.528799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d8520-3817-43f9-807d-cb01d8c135b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:03.529444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d8520-3817-43f9-807d-cb01d8c135b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:03.529800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d8520-3817-43f9-807d-cb01d8c135b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:03.530081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d8520-3817-43f9-807d-cb01d8c135b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:03.530802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d8520-3817-43f9-807d-cb01d8c135b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:03.532775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d8520-3817-43f9-807d-cb01d8c135b9 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:03.564752 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1076/2158] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:46:03 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:46:03.569810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fd2c8-ef3b-4ea8-b589-32474f79b07f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:03.583147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fd2c8-ef3b-4ea8-b589-32474f79b07f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:03.658328 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431d23e-8e7a-4b53-b4fc-6d656a8ac9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:03.658984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431d23e-8e7a-4b53-b4fc-6d656a8ac9d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:03.660191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431d23e-8e7a-4b53-b4fc-6d656a8ac9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:03.660489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431d23e-8e7a-4b53-b4fc-6d656a8ac9d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:03.662988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431d23e-8e7a-4b53-b4fc-6d656a8ac9d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:03.665197 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431d23e-8e7a-4b53-b4fc-6d656a8ac9d0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:03.696667 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1083/2159] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:46:03 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:46:03.697930 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d8520-3817-43f9-807d-cb01d8c135b9 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:03.716012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d8520-3817-43f9-807d-cb01d8c135b9 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:03.716012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b5ce7-ad11-4bd5-99e5-d81b871a2d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:03.716764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b5ce7-ad11-4bd5-99e5-d81b871a2d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:03.717113 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b5ce7-ad11-4bd5-99e5-d81b871a2d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:03.717515 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b5ce7-ad11-4bd5-99e5-d81b871a2d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:03.718323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b5ce7-ad11-4bd5-99e5-d81b871a2d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:03.720100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b5ce7-ad11-4bd5-99e5-d81b871a2d85 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:03.732699 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1077/2160] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:46:03.737826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431d23e-8e7a-4b53-b4fc-6d656a8ac9d0 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:03.756914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431d23e-8e7a-4b53-b4fc-6d656a8ac9d0 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:03.761968 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d141f-fd8c-4016-85f7-3f5104b8f8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:03.762668 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d141f-fd8c-4016-85f7-3f5104b8f8ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:03.762950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d141f-fd8c-4016-85f7-3f5104b8f8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:03.763228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d141f-fd8c-4016-85f7-3f5104b8f8ff admin admin] PATH_INFO: `/v3/projects/2aa5aa8c5cbb4f39bc2d5e906e652a68/users/d415c93f716944868fdf940e0fa088f8/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:03.780849 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d141f-fd8c-4016-85f7-3f5104b8f8ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa5aa8c5cbb4f39bc2d5e906e652a68, user_id=d415c93f716944868fdf940e0fa088f8, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:03.783770 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d141f-fd8c-4016-85f7-3f5104b8f8ff admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:03.800554 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1084/2161] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:46:03 2022] PUT /identity/v3/projects/2aa5aa8c5cbb4f39bc2d5e906e652a68/users/d415c93f716944868fdf940e0fa088f8/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:03.805513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b5ce7-ad11-4bd5-99e5-d81b871a2d85 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:03.834898 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b5ce7-ad11-4bd5-99e5-d81b871a2d85 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:03.957151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7765d0a-0d08-4626-96ba-f840d7c9a9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:03.958061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7765d0a-0d08-4626-96ba-f840d7c9a9cd admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:03.958540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7765d0a-0d08-4626-96ba-f840d7c9a9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:03.958939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7765d0a-0d08-4626-96ba-f840d7c9a9cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:03.959747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7765d0a-0d08-4626-96ba-f840d7c9a9cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:03.965780 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7765d0a-0d08-4626-96ba-f840d7c9a9cd admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:03.982796 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1078/2162] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:46:03.988508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d141f-fd8c-4016-85f7-3f5104b8f8ff admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:04.003978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d141f-fd8c-4016-85f7-3f5104b8f8ff admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:04.010600 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c13b3-3fa2-4d86-a5ff-2370163565e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:04.011479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c13b3-3fa2-4d86-a5ff-2370163565e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:04.011884 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c13b3-3fa2-4d86-a5ff-2370163565e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:04.012268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c13b3-3fa2-4d86-a5ff-2370163565e9 admin admin] PATH_INFO: `/v3/projects/2aa5aa8c5cbb4f39bc2d5e906e652a68/users/d415c93f716944868fdf940e0fa088f8/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:04.015151 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c13b3-3fa2-4d86-a5ff-2370163565e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa5aa8c5cbb4f39bc2d5e906e652a68, user_id=d415c93f716944868fdf940e0fa088f8, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:04.019213 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c13b3-3fa2-4d86-a5ff-2370163565e9 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:04.041805 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1085/2163] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:46:03 2022] PUT /identity/v3/projects/2aa5aa8c5cbb4f39bc2d5e906e652a68/users/d415c93f716944868fdf940e0fa088f8/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:04.043368 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad7903-ad6c-4c11-8872-40701c3110c2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:04.044031 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad7903-ad6c-4c11-8872-40701c3110c2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:04.044419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad7903-ad6c-4c11-8872-40701c3110c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:04.100491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-07ad7903-ad6c-4c11-8872-40701c3110c2 None None] MFA Rules not processed for user `d415c93f716944868fdf940e0fa088f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:04.102574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-07ad7903-ad6c-4c11-8872-40701c3110c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:04.213281 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:04.245772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1079/2164] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:04 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:04.258981 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c13b3-3fa2-4d86-a5ff-2370163565e9 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:04.260973 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9b8c13b3-3fa2-4d86-a5ff-2370163565e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:04.312954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:04.342271 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c13b3-3fa2-4d86-a5ff-2370163565e9 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:04.346069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51585e76-7bcc-422c-9234-f10c144b56cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:04.347222 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51585e76-7bcc-422c-9234-f10c144b56cd service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:04.347534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51585e76-7bcc-422c-9234-f10c144b56cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:04.347764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51585e76-7bcc-422c-9234-f10c144b56cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:04.357759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51585e76-7bcc-422c-9234-f10c144b56cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:04.359462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51585e76-7bcc-422c-9234-f10c144b56cd service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:04.371298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1086/2165] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:46:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:46:04.459640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad7903-ad6c-4c11-8872-40701c3110c2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:04.472457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad7903-ad6c-4c11-8872-40701c3110c2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:04.476130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86f0cc-03ca-4df0-b2a7-d6e308c14074 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:04.476859 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86f0cc-03ca-4df0-b2a7-d6e308c14074 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:04.477213 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86f0cc-03ca-4df0-b2a7-d6e308c14074 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:04.477595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86f0cc-03ca-4df0-b2a7-d6e308c14074 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:04.543024 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86f0cc-03ca-4df0-b2a7-d6e308c14074 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:04.544745 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86f0cc-03ca-4df0-b2a7-d6e308c14074 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:04.556535 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1080/2166] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:46:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:46:09.557276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51585e76-7bcc-422c-9234-f10c144b56cd service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:09.632940 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51585e76-7bcc-422c-9234-f10c144b56cd service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:09.639662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4faa6b3-05ad-4636-9367-0017dd077efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:09.640577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4faa6b3-05ad-4636-9367-0017dd077efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:09.640577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4faa6b3-05ad-4636-9367-0017dd077efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:09.641006 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4faa6b3-05ad-4636-9367-0017dd077efb admin admin] PATH_INFO: `/v3/users/7175291123eb4eceaa3b17aec9cf3e3e` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:09.642232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4faa6b3-05ad-4636-9367-0017dd077efb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7175291123eb4eceaa3b17aec9cf3e3e)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:09.649349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4faa6b3-05ad-4636-9367-0017dd077efb admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:09.706155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c4faa6b3-05ad-4636-9367-0017dd077efb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7175291123eb4eceaa3b17aec9cf3e3e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:09.719475 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c4faa6b3-05ad-4636-9367-0017dd077efb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7175291123eb4eceaa3b17aec9cf3e3e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:09.720233 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c4faa6b3-05ad-4636-9367-0017dd077efb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7175291123eb4eceaa3b17aec9cf3e3e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:09.730184 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c4faa6b3-05ad-4636-9367-0017dd077efb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7175291123eb4eceaa3b17aec9cf3e3e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:09.744968 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c4faa6b3-05ad-4636-9367-0017dd077efb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7175291123eb4eceaa3b17aec9cf3e3e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:09.751651 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1087/2167] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:46:09 2022] DELETE /identity/v3/users/7175291123eb4eceaa3b17aec9cf3e3e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:09.756854 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb03e0-99a4-48cb-aa9c-41ef647527ab None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:09.758590 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb03e0-99a4-48cb-aa9c-41ef647527ab None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:09.758590 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb03e0-99a4-48cb-aa9c-41ef647527ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:09.779030 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-34fb03e0-99a4-48cb-aa9c-41ef647527ab None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:09.780250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-34fb03e0-99a4-48cb-aa9c-41ef647527ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:09.844877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1081/2168] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:09.855337 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4faa6b3-05ad-4636-9367-0017dd077efb admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:09.856816 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c4faa6b3-05ad-4636-9367-0017dd077efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:09.931592 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4faa6b3-05ad-4636-9367-0017dd077efb admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:09.935786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7976732-acb9-474a-85f6-278b08562a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:09.937295 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7976732-acb9-474a-85f6-278b08562a44 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:09.937636 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7976732-acb9-474a-85f6-278b08562a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:09.937919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7976732-acb9-474a-85f6-278b08562a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:09.946632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:09.949495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7976732-acb9-474a-85f6-278b08562a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:09.950896 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7976732-acb9-474a-85f6-278b08562a44 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:09.963221 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1088/2169] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:46:10.406196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb03e0-99a4-48cb-aa9c-41ef647527ab None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:10.407625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-34fb03e0-99a4-48cb-aa9c-41ef647527ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:10.420315 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb03e0-99a4-48cb-aa9c-41ef647527ab None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:10.425873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:10.426531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:10.426813 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:10.427088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e admin admin] PATH_INFO: `/v3/projects/2332cd679d6c429d9ddcb39948bd4914` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:10.428277 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2332cd679d6c429d9ddcb39948bd4914)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:10.430359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:10.498067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2332cd679d6c429d9ddcb39948bd4914', 'request_id': 'req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:10.510086 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2332cd679d6c429d9ddcb39948bd4914', 'request_id': 'req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:10.522577 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2332cd679d6c429d9ddcb39948bd4914', 'request_id': 'req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:10.535942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e admin admin] The token cache is being invalidate because project 2332cd679d6c429d9ddcb39948bd4914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:46:10.536237 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:10.536944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:10.538564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1082/2170] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:46:10 2022] DELETE /identity/v3/projects/2332cd679d6c429d9ddcb39948bd4914 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:10.544318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7976732-acb9-474a-85f6-278b08562a44 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:10.555693 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7976732-acb9-474a-85f6-278b08562a44 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:10.625707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2820f-61cb-4fca-bf17-fafd72825013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:10.626426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2820f-61cb-4fca-bf17-fafd72825013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:10.626717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2820f-61cb-4fca-bf17-fafd72825013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:10.627024 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2820f-61cb-4fca-bf17-fafd72825013 admin admin] PATH_INFO: `/v3/users/d415c93f716944868fdf940e0fa088f8` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:10.628855 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2820f-61cb-4fca-bf17-fafd72825013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d415c93f716944868fdf940e0fa088f8)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:10.630935 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2820f-61cb-4fca-bf17-fafd72825013 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:10.695157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-94c2820f-61cb-4fca-bf17-fafd72825013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd415c93f716944868fdf940e0fa088f8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:10.712767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-94c2820f-61cb-4fca-bf17-fafd72825013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd415c93f716944868fdf940e0fa088f8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:10.713659 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-94c2820f-61cb-4fca-bf17-fafd72825013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd415c93f716944868fdf940e0fa088f8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:10.723998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-94c2820f-61cb-4fca-bf17-fafd72825013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd415c93f716944868fdf940e0fa088f8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:10.737892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-94c2820f-61cb-4fca-bf17-fafd72825013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd415c93f716944868fdf940e0fa088f8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:10.748266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1089/2171] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:46:10 2022] DELETE /identity/v3/users/d415c93f716944868fdf940e0fa088f8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:11.178167 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:11.182277 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:11.210698 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6ea7b-1972-47f7-b0fb-e63926bbc05e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:11.284714 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:11.425822 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999aaf75-f68c-4f26-8c52-9c80a29dd216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:11.426540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999aaf75-f68c-4f26-8c52-9c80a29dd216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:11.426805 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999aaf75-f68c-4f26-8c52-9c80a29dd216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:11.427084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999aaf75-f68c-4f26-8c52-9c80a29dd216 admin admin] PATH_INFO: `/v3/projects/2aa5aa8c5cbb4f39bc2d5e906e652a68` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:11.432326 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999aaf75-f68c-4f26-8c52-9c80a29dd216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aa5aa8c5cbb4f39bc2d5e906e652a68)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:11.439171 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999aaf75-f68c-4f26-8c52-9c80a29dd216 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:11.559178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-999aaf75-f68c-4f26-8c52-9c80a29dd216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aa5aa8c5cbb4f39bc2d5e906e652a68', 'request_id': 'req-999aaf75-f68c-4f26-8c52-9c80a29dd216'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:11.572927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-999aaf75-f68c-4f26-8c52-9c80a29dd216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aa5aa8c5cbb4f39bc2d5e906e652a68', 'request_id': 'req-999aaf75-f68c-4f26-8c52-9c80a29dd216'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:11.589762 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-999aaf75-f68c-4f26-8c52-9c80a29dd216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aa5aa8c5cbb4f39bc2d5e906e652a68', 'request_id': 'req-999aaf75-f68c-4f26-8c52-9c80a29dd216'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:11.640116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-999aaf75-f68c-4f26-8c52-9c80a29dd216 admin admin] The token cache is being invalidate because project 2aa5aa8c5cbb4f39bc2d5e906e652a68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:46:11.640464 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-999aaf75-f68c-4f26-8c52-9c80a29dd216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:11.644674 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-999aaf75-f68c-4f26-8c52-9c80a29dd216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:11.653022 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1083/2172] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:46:11 2022] DELETE /identity/v3/projects/2aa5aa8c5cbb4f39bc2d5e906e652a68 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:11.655547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c49e40-27f6-450a-acb1-878b203befe6 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:11.656197 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c49e40-27f6-450a-acb1-878b203befe6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:11.656197 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c49e40-27f6-450a-acb1-878b203befe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:11.701186 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-58c49e40-27f6-450a-acb1-878b203befe6 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:11.703227 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-58c49e40-27f6-450a-acb1-878b203befe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:11.837938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34b583-e91e-460c-ad51-1fa4f077637d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:11.838237 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34b583-e91e-460c-ad51-1fa4f077637d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:11.838528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34b583-e91e-460c-ad51-1fa4f077637d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:11.855821 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1090/2173] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:46:11.901852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5f34b583-e91e-460c-ad51-1fa4f077637d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:11.903634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5f34b583-e91e-460c-ad51-1fa4f077637d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:11.915903 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1084/2174] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:11.917273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c49e40-27f6-450a-acb1-878b203befe6 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:11.929142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c49e40-27f6-450a-acb1-878b203befe6 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:11.937712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24a543-554b-4fc4-a33d-65460a6a5e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:11.938409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24a543-554b-4fc4-a33d-65460a6a5e3e admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:11.938650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24a543-554b-4fc4-a33d-65460a6a5e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:11.938912 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24a543-554b-4fc4-a33d-65460a6a5e3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:11.939630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24a543-554b-4fc4-a33d-65460a6a5e3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:46:11.939935 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24a543-554b-4fc4-a33d-65460a6a5e3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:11.941904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24a543-554b-4fc4-a33d-65460a6a5e3e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:11.965307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1091/2175] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:46:11.970000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34b583-e91e-460c-ad51-1fa4f077637d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:11.992016 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34b583-e91e-460c-ad51-1fa4f077637d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:12.000553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da9f80-25d7-451e-8918-de96efcb2021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:12.001844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da9f80-25d7-451e-8918-de96efcb2021 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:12.002270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da9f80-25d7-451e-8918-de96efcb2021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:12.002980 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da9f80-25d7-451e-8918-de96efcb2021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:12.004254 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da9f80-25d7-451e-8918-de96efcb2021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:12.007625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da9f80-25d7-451e-8918-de96efcb2021 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:12.078314 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1085/2176] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:46:11 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:46:12.085604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24a543-554b-4fc4-a33d-65460a6a5e3e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:12.101571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:12.114946 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24a543-554b-4fc4-a33d-65460a6a5e3e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:12.242099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db08b17-c386-496b-9a8a-7c30f145e61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:12.242866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db08b17-c386-496b-9a8a-7c30f145e61e admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:12.243145 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db08b17-c386-496b-9a8a-7c30f145e61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:12.243513 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db08b17-c386-496b-9a8a-7c30f145e61e admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:12.244307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db08b17-c386-496b-9a8a-7c30f145e61e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:12.246813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db08b17-c386-496b-9a8a-7c30f145e61e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:12.306994 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1092/2177] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:46:12 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:46:12.312456 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da9f80-25d7-451e-8918-de96efcb2021 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:12.341438 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da9f80-25d7-451e-8918-de96efcb2021 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:12.350374 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270babd-c84c-4252-85e6-f80a8788cc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:12.351146 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270babd-c84c-4252-85e6-f80a8788cc89 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:12.351494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270babd-c84c-4252-85e6-f80a8788cc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:12.351835 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270babd-c84c-4252-85e6-f80a8788cc89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:12.353359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270babd-c84c-4252-85e6-f80a8788cc89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:12.355131 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270babd-c84c-4252-85e6-f80a8788cc89 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:12.374057 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1086/2178] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:46:12.375610 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db08b17-c386-496b-9a8a-7c30f145e61e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:12.389063 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db08b17-c386-496b-9a8a-7c30f145e61e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:12.395530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0b3e9-203b-4250-9e46-71a08aa50d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:12.396455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0b3e9-203b-4250-9e46-71a08aa50d9e admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:12.396864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0b3e9-203b-4250-9e46-71a08aa50d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:12.397423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0b3e9-203b-4250-9e46-71a08aa50d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:12.398349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0b3e9-203b-4250-9e46-71a08aa50d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:12.405090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0b3e9-203b-4250-9e46-71a08aa50d9e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:12.425515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1093/2179] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:46:12.436289 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270babd-c84c-4252-85e6-f80a8788cc89 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:12.451028 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270babd-c84c-4252-85e6-f80a8788cc89 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:12.458542 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e53c76-4fff-42b9-a037-5f70a6076192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:12.459752 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53c76-4fff-42b9-a037-5f70a6076192 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:12.460268 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53c76-4fff-42b9-a037-5f70a6076192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:12.460815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53c76-4fff-42b9-a037-5f70a6076192 admin admin] PATH_INFO: `/v3/projects/15c056a479c249f39ee486b7055a6d1a/users/14a9e7c838164b12949b1ab1f6c86bc8/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:12.480682 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e53c76-4fff-42b9-a037-5f70a6076192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15c056a479c249f39ee486b7055a6d1a, user_id=14a9e7c838164b12949b1ab1f6c86bc8, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:12.483375 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e53c76-4fff-42b9-a037-5f70a6076192 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:12.498441 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1087/2180] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:46:12 2022] PUT /identity/v3/projects/15c056a479c249f39ee486b7055a6d1a/users/14a9e7c838164b12949b1ab1f6c86bc8/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:12.510147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6c048-0daa-41f3-8117-99a691bd8c29 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:12.510487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6c048-0daa-41f3-8117-99a691bd8c29 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:12.510804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6c048-0daa-41f3-8117-99a691bd8c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:12.550422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-51c6c048-0daa-41f3-8117-99a691bd8c29 None None] MFA Rules not processed for user `14a9e7c838164b12949b1ab1f6c86bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:12.552192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-51c6c048-0daa-41f3-8117-99a691bd8c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:12.649374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1094/2181] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:12.660652 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e53c76-4fff-42b9-a037-5f70a6076192 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:12.663049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f1e53c76-4fff-42b9-a037-5f70a6076192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:12.738450 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:12.758961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e53c76-4fff-42b9-a037-5f70a6076192 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:12.767430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d767e-529e-4f29-b6f6-d5059d129797 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:12.768081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d767e-529e-4f29-b6f6-d5059d129797 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:12.768425 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d767e-529e-4f29-b6f6-d5059d129797 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:12.769202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d767e-529e-4f29-b6f6-d5059d129797 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:12.779939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d767e-529e-4f29-b6f6-d5059d129797 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:12.781448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d767e-529e-4f29-b6f6-d5059d129797 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:12.792878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1088/2182] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:46:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:46:22.640205 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:22.788192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=7 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:24.994540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6c048-0daa-41f3-8117-99a691bd8c29 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:24.996509 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-51c6c048-0daa-41f3-8117-99a691bd8c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:25.146291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6c048-0daa-41f3-8117-99a691bd8c29 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:25.161759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91117e47-b7bd-4688-83aa-62e26f2cef75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:25.162500 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91117e47-b7bd-4688-83aa-62e26f2cef75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:25.162933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91117e47-b7bd-4688-83aa-62e26f2cef75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:25.163352 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91117e47-b7bd-4688-83aa-62e26f2cef75 admin admin] PATH_INFO: `/v3/users/2f336db03c2d48cf8e9be0eaf3578f18` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:25.165420 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91117e47-b7bd-4688-83aa-62e26f2cef75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f336db03c2d48cf8e9be0eaf3578f18)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:25.169881 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91117e47-b7bd-4688-83aa-62e26f2cef75 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:25.293180 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-91117e47-b7bd-4688-83aa-62e26f2cef75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f336db03c2d48cf8e9be0eaf3578f18'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:25.317310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-91117e47-b7bd-4688-83aa-62e26f2cef75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f336db03c2d48cf8e9be0eaf3578f18'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:25.318154 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-91117e47-b7bd-4688-83aa-62e26f2cef75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f336db03c2d48cf8e9be0eaf3578f18'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:25.329127 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-91117e47-b7bd-4688-83aa-62e26f2cef75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f336db03c2d48cf8e9be0eaf3578f18'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:25.337574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-91117e47-b7bd-4688-83aa-62e26f2cef75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f336db03c2d48cf8e9be0eaf3578f18'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:25.344873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1095/2183] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:46:24 2022] DELETE /identity/v3/users/2f336db03c2d48cf8e9be0eaf3578f18 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:25.350613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7f5cc-cfc4-4de1-8f9b-c222aafe010f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:25.351167 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7f5cc-cfc4-4de1-8f9b-c222aafe010f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:25.351167 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7f5cc-cfc4-4de1-8f9b-c222aafe010f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:25.373978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b1f7f5cc-cfc4-4de1-8f9b-c222aafe010f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:25.376276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b1f7f5cc-cfc4-4de1-8f9b-c222aafe010f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:25.460623 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1089/2184] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:25.470930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91117e47-b7bd-4688-83aa-62e26f2cef75 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:25.473613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-91117e47-b7bd-4688-83aa-62e26f2cef75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:25.566701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91117e47-b7bd-4688-83aa-62e26f2cef75 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:25.586828 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf9566-0ba9-4f5d-b5b9-397375b08e5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:25.587640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf9566-0ba9-4f5d-b5b9-397375b08e5c service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:25.587930 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf9566-0ba9-4f5d-b5b9-397375b08e5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:25.588326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf9566-0ba9-4f5d-b5b9-397375b08e5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:25.613397 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf9566-0ba9-4f5d-b5b9-397375b08e5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:25.616015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf9566-0ba9-4f5d-b5b9-397375b08e5c service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:25.627940 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1096/2185] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:46:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:46:26.431154 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f8553-40c6-4069-8cd1-889f214ee16d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:26.432061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f8553-40c6-4069-8cd1-889f214ee16d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:26.432538 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f8553-40c6-4069-8cd1-889f214ee16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:26.453621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d25f8553-40c6-4069-8cd1-889f214ee16d None None] MFA Rules not processed for user `a5e714aad25f4fda8acf767ad7e2490b`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:26.455244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d25f8553-40c6-4069-8cd1-889f214ee16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:26.541820 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1090/2186] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:26.717268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf9566-0ba9-4f5d-b5b9-397375b08e5c service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:26.719526 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9bbf9566-0ba9-4f5d-b5b9-397375b08e5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:26.738571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf9566-0ba9-4f5d-b5b9-397375b08e5c service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:26.743966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafede0-17d7-4e0c-af65-62a54e05f56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:26.745306 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafede0-17d7-4e0c-af65-62a54e05f56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:26.746348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafede0-17d7-4e0c-af65-62a54e05f56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:26.746348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafede0-17d7-4e0c-af65-62a54e05f56d admin admin] PATH_INFO: `/v3/projects/cf86280b8a364172ab65111a257a8bb6` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:26.747832 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafede0-17d7-4e0c-af65-62a54e05f56d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf86280b8a364172ab65111a257a8bb6)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:26.750349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafede0-17d7-4e0c-af65-62a54e05f56d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:26.777113 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:26.834959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-faafede0-17d7-4e0c-af65-62a54e05f56d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf86280b8a364172ab65111a257a8bb6', 'request_id': 'req-faafede0-17d7-4e0c-af65-62a54e05f56d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:26.853555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-faafede0-17d7-4e0c-af65-62a54e05f56d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf86280b8a364172ab65111a257a8bb6', 'request_id': 'req-faafede0-17d7-4e0c-af65-62a54e05f56d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:26.867226 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-faafede0-17d7-4e0c-af65-62a54e05f56d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf86280b8a364172ab65111a257a8bb6', 'request_id': 'req-faafede0-17d7-4e0c-af65-62a54e05f56d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:26.883341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-faafede0-17d7-4e0c-af65-62a54e05f56d admin admin] The token cache is being invalidate because project cf86280b8a364172ab65111a257a8bb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:46:26.884259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-faafede0-17d7-4e0c-af65-62a54e05f56d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:26.885174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-faafede0-17d7-4e0c-af65-62a54e05f56d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:26.888951 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1097/2187] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:46:26 2022] DELETE /identity/v3/projects/cf86280b8a364172ab65111a257a8bb6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:26.895095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07003b8c-2b05-4e5e-924e-20e8fd7915a1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:26.897017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07003b8c-2b05-4e5e-924e-20e8fd7915a1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:26.897651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07003b8c-2b05-4e5e-924e-20e8fd7915a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:26.929978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-07003b8c-2b05-4e5e-924e-20e8fd7915a1 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:26.931622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-07003b8c-2b05-4e5e-924e-20e8fd7915a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:27.094513 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5139332-1953-4479-a02d-ded08ee29552 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:27.095050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5139332-1953-4479-a02d-ded08ee29552 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:27.095440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5139332-1953-4479-a02d-ded08ee29552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:27.097192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1091/2188] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:27.138027 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a5139332-1953-4479-a02d-ded08ee29552 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:27.139330 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a5139332-1953-4479-a02d-ded08ee29552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:27.169745 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1098/2189] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:46:27.173530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07003b8c-2b05-4e5e-924e-20e8fd7915a1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:27.189348 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07003b8c-2b05-4e5e-924e-20e8fd7915a1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:27.196428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d3067-2345-4a1f-ac8d-c448b30d34e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:27.198562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d3067-2345-4a1f-ac8d-c448b30d34e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:27.199549 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d3067-2345-4a1f-ac8d-c448b30d34e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:27.200152 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d3067-2345-4a1f-ac8d-c448b30d34e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:27.201347 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d3067-2345-4a1f-ac8d-c448b30d34e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:46:27.202020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d3067-2345-4a1f-ac8d-c448b30d34e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:27.204503 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d3067-2345-4a1f-ac8d-c448b30d34e0 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:27.222381 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1092/2190] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:46:27.226966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5139332-1953-4479-a02d-ded08ee29552 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:27.237867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5139332-1953-4479-a02d-ded08ee29552 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:27.243418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bc87b-1ca0-4058-b0e5-47eb475f4a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:27.243904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bc87b-1ca0-4058-b0e5-47eb475f4a46 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:27.244304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bc87b-1ca0-4058-b0e5-47eb475f4a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:27.244674 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bc87b-1ca0-4058-b0e5-47eb475f4a46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:27.245443 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bc87b-1ca0-4058-b0e5-47eb475f4a46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:27.247480 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bc87b-1ca0-4058-b0e5-47eb475f4a46 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:27.284741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1099/2191] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:46:27 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:46:27.288098 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d3067-2345-4a1f-ac8d-c448b30d34e0 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:27.300697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d3067-2345-4a1f-ac8d-c448b30d34e0 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:27.357759 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:27.397070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb9f52-1204-4583-b352-ca98744f9ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:27.398013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb9f52-1204-4583-b352-ca98744f9ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:27.398300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb9f52-1204-4583-b352-ca98744f9ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:27.398577 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb9f52-1204-4583-b352-ca98744f9ad4 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:27.399443 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb9f52-1204-4583-b352-ca98744f9ad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:27.401496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb9f52-1204-4583-b352-ca98744f9ad4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:27.436161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bc87b-1ca0-4058-b0e5-47eb475f4a46 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:27.447757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1093/2192] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:46:27 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:46:27.451438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bc87b-1ca0-4058-b0e5-47eb475f4a46 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:27.471996 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240f4e7-f388-4669-9051-dbb80cbc9c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:27.472567 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240f4e7-f388-4669-9051-dbb80cbc9c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:27.472846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240f4e7-f388-4669-9051-dbb80cbc9c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:27.473114 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240f4e7-f388-4669-9051-dbb80cbc9c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:27.473901 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240f4e7-f388-4669-9051-dbb80cbc9c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:27.475581 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240f4e7-f388-4669-9051-dbb80cbc9c3e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:27.494093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb9f52-1204-4583-b352-ca98744f9ad4 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:27.504954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1100/2193] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:46:27.510722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb9f52-1204-4583-b352-ca98744f9ad4 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:27.517026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616748e-eb77-44a5-ac7a-c927162ea513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:27.517892 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616748e-eb77-44a5-ac7a-c927162ea513 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:27.518240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616748e-eb77-44a5-ac7a-c927162ea513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:27.519164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616748e-eb77-44a5-ac7a-c927162ea513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:27.520101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616748e-eb77-44a5-ac7a-c927162ea513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:27.521967 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616748e-eb77-44a5-ac7a-c927162ea513 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:27.545886 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240f4e7-f388-4669-9051-dbb80cbc9c3e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:27.558092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1094/2194] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:46:27.561261 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240f4e7-f388-4669-9051-dbb80cbc9c3e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:27.566756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb3ee5-9422-4927-9a73-f4e636b7444c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:27.567536 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb3ee5-9422-4927-9a73-f4e636b7444c admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:27.567886 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb3ee5-9422-4927-9a73-f4e636b7444c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:27.568175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb3ee5-9422-4927-9a73-f4e636b7444c admin admin] PATH_INFO: `/v3/projects/f9d4304db6d342e4ada925b6784e6d0d/users/92485e486d53488eabf12105d1048e13/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:27.586870 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb3ee5-9422-4927-9a73-f4e636b7444c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d4304db6d342e4ada925b6784e6d0d, user_id=92485e486d53488eabf12105d1048e13, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:27.589699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb3ee5-9422-4927-9a73-f4e636b7444c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:27.624138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1101/2195] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:46:27 2022] PUT /identity/v3/projects/f9d4304db6d342e4ada925b6784e6d0d/users/92485e486d53488eabf12105d1048e13/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:27.630303 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae2e44-e0ed-4709-8735-7dd34996dfda None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:27.630651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae2e44-e0ed-4709-8735-7dd34996dfda None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:27.630862 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae2e44-e0ed-4709-8735-7dd34996dfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:27.654444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-88ae2e44-e0ed-4709-8735-7dd34996dfda None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:27.655777 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-88ae2e44-e0ed-4709-8735-7dd34996dfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:27.826430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1095/2196] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:27.832404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb3ee5-9422-4927-9a73-f4e636b7444c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:27.834383 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9ceb3ee5-9422-4927-9a73-f4e636b7444c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:27.960841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb3ee5-9422-4927-9a73-f4e636b7444c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:27.968045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81bcee7-a2a8-47a6-baf7-6574354dee47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:27.968923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bcee7-a2a8-47a6-baf7-6574354dee47 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:27.970595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bcee7-a2a8-47a6-baf7-6574354dee47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:27.970595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bcee7-a2a8-47a6-baf7-6574354dee47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:27.991950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81bcee7-a2a8-47a6-baf7-6574354dee47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:27.994489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81bcee7-a2a8-47a6-baf7-6574354dee47 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:28.009719 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1102/2197] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:46:28.442676 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc05728-0f78-45ec-ad7a-fc0de0084785 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:28.443736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc05728-0f78-45ec-ad7a-fc0de0084785 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:28.444226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc05728-0f78-45ec-ad7a-fc0de0084785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:28.481010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4fc05728-0f78-45ec-ad7a-fc0de0084785 None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:28.484794 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4fc05728-0f78-45ec-ad7a-fc0de0084785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:28.705567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1096/2198] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 3 10:46:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:28.803155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81bcee7-a2a8-47a6-baf7-6574354dee47 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:28.807128 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d81bcee7-a2a8-47a6-baf7-6574354dee47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:28.988651 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:29.091479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81bcee7-a2a8-47a6-baf7-6574354dee47 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:29.103774 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab6e96-3a6e-4109-a560-25894e55ad74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:29.105753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab6e96-3a6e-4109-a560-25894e55ad74 service swift] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:29.106625 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab6e96-3a6e-4109-a560-25894e55ad74 service swift] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:29.107392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab6e96-3a6e-4109-a560-25894e55ad74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:29.130551 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab6e96-3a6e-4109-a560-25894e55ad74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:29.133791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab6e96-3a6e-4109-a560-25894e55ad74 service swift] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:29.144906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1103/2199] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:46:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 10:46:29.715055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0d86c-4b28-4da2-8d80-cd6d24f151ea None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:29.715055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0d86c-4b28-4da2-8d80-cd6d24f151ea None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:29.715632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0d86c-4b28-4da2-8d80-cd6d24f151ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:29.756971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-63e0d86c-4b28-4da2-8d80-cd6d24f151ea None None] MFA Rules not processed for user `92485e486d53488eabf12105d1048e13`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:29.758627 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-63e0d86c-4b28-4da2-8d80-cd6d24f151ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:29.849020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:29.897746 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1097/2200] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:29.911410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab6e96-3a6e-4109-a560-25894e55ad74 service swift] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:29.912964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-82ab6e96-3a6e-4109-a560-25894e55ad74 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:30.047222 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab6e96-3a6e-4109-a560-25894e55ad74 service swift] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:30.059767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5513c7-b0ac-4443-b414-c649117bf2cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:30.060718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5513c7-b0ac-4443-b414-c649117bf2cc service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:30.061325 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5513c7-b0ac-4443-b414-c649117bf2cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:30.061848 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5513c7-b0ac-4443-b414-c649117bf2cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:30.079452 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5513c7-b0ac-4443-b414-c649117bf2cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:30.082688 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5513c7-b0ac-4443-b414-c649117bf2cc service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:30.107293 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1104/2201] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:46:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:46:33.010913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9877-78bf-479c-9952-9cb5eb0a8c6e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:33.012295 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9877-78bf-479c-9952-9cb5eb0a8c6e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:33.012999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9877-78bf-479c-9952-9cb5eb0a8c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:33.051287 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f9ec9877-78bf-479c-9952-9cb5eb0a8c6e None None] MFA Rules not processed for user `92485e486d53488eabf12105d1048e13`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:33.052576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f9ec9877-78bf-479c-9952-9cb5eb0a8c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:33.066878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1098/2202] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:37.632614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5513c7-b0ac-4443-b414-c649117bf2cc service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:37.634908 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bf5513c7-b0ac-4443-b414-c649117bf2cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:37.695574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5513c7-b0ac-4443-b414-c649117bf2cc service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:37.713847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786531b2-b81c-4bbc-af4e-481a710cc8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:37.716891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786531b2-b81c-4bbc-af4e-481a710cc8a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:37.717381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786531b2-b81c-4bbc-af4e-481a710cc8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:37.717699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786531b2-b81c-4bbc-af4e-481a710cc8a8 admin admin] PATH_INFO: `/v3/users/14a9e7c838164b12949b1ab1f6c86bc8` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:37.719778 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786531b2-b81c-4bbc-af4e-481a710cc8a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14a9e7c838164b12949b1ab1f6c86bc8)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:37.723036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786531b2-b81c-4bbc-af4e-481a710cc8a8 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:37.836367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-786531b2-b81c-4bbc-af4e-481a710cc8a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14a9e7c838164b12949b1ab1f6c86bc8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:37.861569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-786531b2-b81c-4bbc-af4e-481a710cc8a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14a9e7c838164b12949b1ab1f6c86bc8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:37.862490 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-786531b2-b81c-4bbc-af4e-481a710cc8a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14a9e7c838164b12949b1ab1f6c86bc8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:37.878301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-786531b2-b81c-4bbc-af4e-481a710cc8a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14a9e7c838164b12949b1ab1f6c86bc8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:37.904164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-786531b2-b81c-4bbc-af4e-481a710cc8a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14a9e7c838164b12949b1ab1f6c86bc8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:37.919337 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1105/2203] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:46:37 2022] DELETE /identity/v3/users/14a9e7c838164b12949b1ab1f6c86bc8 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:46:37.922151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d50eb-1d55-4525-a34c-ca99f683e2be None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:37.922739 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d50eb-1d55-4525-a34c-ca99f683e2be None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:37.923299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d50eb-1d55-4525-a34c-ca99f683e2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:37.976253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4b6d50eb-1d55-4525-a34c-ca99f683e2be None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:37.979407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4b6d50eb-1d55-4525-a34c-ca99f683e2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:38.152180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1099/2204] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:46:38.166919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786531b2-b81c-4bbc-af4e-481a710cc8a8 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:38.170245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-786531b2-b81c-4bbc-af4e-481a710cc8a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:38.322671 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:38.337822 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786531b2-b81c-4bbc-af4e-481a710cc8a8 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:38.349742 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c4604-2d77-4705-9063-1671da521c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:38.350314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c4604-2d77-4705-9063-1671da521c67 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:38.350627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c4604-2d77-4705-9063-1671da521c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:38.350963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c4604-2d77-4705-9063-1671da521c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:38.377533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c4604-2d77-4705-9063-1671da521c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:38.379883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c4604-2d77-4705-9063-1671da521c67 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:38.426490 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1106/2205] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:46:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:46:38.776041 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d50eb-1d55-4525-a34c-ca99f683e2be None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:38.777782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4b6d50eb-1d55-4525-a34c-ca99f683e2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:38.960397 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d50eb-1d55-4525-a34c-ca99f683e2be None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:38.970809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15a685-2108-4c18-8270-e611febcccf7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2c5536eddd346d692822a2f3befb48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:38.972587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15a685-2108-4c18-8270-e611febcccf7 service placement] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:38.973212 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15a685-2108-4c18-8270-e611febcccf7 service placement] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:38.973841 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15a685-2108-4c18-8270-e611febcccf7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:38.977693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fa15a685-2108-4c18-8270-e611febcccf7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:39.022229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15a685-2108-4c18-8270-e611febcccf7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:39.031678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15a685-2108-4c18-8270-e611febcccf7 service placement] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:39.108826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1100/2206] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:46:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:46:40.041202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c4604-2d77-4705-9063-1671da521c67 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:40.043397 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-646c4604-2d77-4705-9063-1671da521c67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:40.067339 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c4604-2d77-4705-9063-1671da521c67 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:40.082533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612a751-ccd0-4b17-beb8-46457e0c9309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:40.082533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612a751-ccd0-4b17-beb8-46457e0c9309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:40.082533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612a751-ccd0-4b17-beb8-46457e0c9309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:40.083147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612a751-ccd0-4b17-beb8-46457e0c9309 admin admin] PATH_INFO: `/v3/projects/15c056a479c249f39ee486b7055a6d1a` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:40.084846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612a751-ccd0-4b17-beb8-46457e0c9309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15c056a479c249f39ee486b7055a6d1a)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:40.087134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612a751-ccd0-4b17-beb8-46457e0c9309 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:40.194989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5612a751-ccd0-4b17-beb8-46457e0c9309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15c056a479c249f39ee486b7055a6d1a', 'request_id': 'req-5612a751-ccd0-4b17-beb8-46457e0c9309'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:40.217531 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5612a751-ccd0-4b17-beb8-46457e0c9309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15c056a479c249f39ee486b7055a6d1a', 'request_id': 'req-5612a751-ccd0-4b17-beb8-46457e0c9309'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:40.239333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5612a751-ccd0-4b17-beb8-46457e0c9309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15c056a479c249f39ee486b7055a6d1a', 'request_id': 'req-5612a751-ccd0-4b17-beb8-46457e0c9309'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:40.268992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5612a751-ccd0-4b17-beb8-46457e0c9309 admin admin] The token cache is being invalidate because project 15c056a479c249f39ee486b7055a6d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:46:40.268992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5612a751-ccd0-4b17-beb8-46457e0c9309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:40.268992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5612a751-ccd0-4b17-beb8-46457e0c9309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:46:40.277592 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1107/2207] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:46:40 2022] DELETE /identity/v3/projects/15c056a479c249f39ee486b7055a6d1a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:40.282977 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66902f1-e29f-44a0-9240-e433c43ca75f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:40.284205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66902f1-e29f-44a0-9240-e433c43ca75f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:40.284944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66902f1-e29f-44a0-9240-e433c43ca75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:40.379709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c66902f1-e29f-44a0-9240-e433c43ca75f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:40.383243 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c66902f1-e29f-44a0-9240-e433c43ca75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:40.616083 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1101/2208] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:40.628467 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d59510-75eb-491c-93e5-6c334a6a9e53 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:40.628773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d59510-75eb-491c-93e5-6c334a6a9e53 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:40.629055 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d59510-75eb-491c-93e5-6c334a6a9e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:40.693571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-23d59510-75eb-491c-93e5-6c334a6a9e53 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:40.693571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-23d59510-75eb-491c-93e5-6c334a6a9e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:40.706793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1108/2209] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:40.709205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66902f1-e29f-44a0-9240-e433c43ca75f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:40.749774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66902f1-e29f-44a0-9240-e433c43ca75f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:40.766758 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515e430-689d-48d9-8b8b-d4aae69d6ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:40.768005 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515e430-689d-48d9-8b8b-d4aae69d6ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:40.768604 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515e430-689d-48d9-8b8b-d4aae69d6ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:40.774354 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515e430-689d-48d9-8b8b-d4aae69d6ea9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:40.775742 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515e430-689d-48d9-8b8b-d4aae69d6ea9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:46:40.776622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515e430-689d-48d9-8b8b-d4aae69d6ea9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:40.783413 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515e430-689d-48d9-8b8b-d4aae69d6ea9 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:40.808540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:40.823445 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1102/2210] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:46:40.825687 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d59510-75eb-491c-93e5-6c334a6a9e53 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:40.900458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d59510-75eb-491c-93e5-6c334a6a9e53 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:40.900458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db2c47-558d-4fa1-b264-b06d46579b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:40.900458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db2c47-558d-4fa1-b264-b06d46579b9d admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:40.900458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db2c47-558d-4fa1-b264-b06d46579b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:40.900978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db2c47-558d-4fa1-b264-b06d46579b9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:40.900978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db2c47-558d-4fa1-b264-b06d46579b9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:40.900978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db2c47-558d-4fa1-b264-b06d46579b9d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:41.056014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1109/2211] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:46:40 2022] POST /identity/v3/projects => generated 365 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:46:41.059791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515e430-689d-48d9-8b8b-d4aae69d6ea9 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:41.086315 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515e430-689d-48d9-8b8b-d4aae69d6ea9 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:41.419247 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25376078-5062-4f61-b2f6-e87ac33813d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:41.420379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25376078-5062-4f61-b2f6-e87ac33813d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:41.420907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25376078-5062-4f61-b2f6-e87ac33813d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:41.421398 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25376078-5062-4f61-b2f6-e87ac33813d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:41.423419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25376078-5062-4f61-b2f6-e87ac33813d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:41.430516 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25376078-5062-4f61-b2f6-e87ac33813d8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:41.494670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1103/2212] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:46:41 2022] POST /identity/v3/users => generated 283 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:46:41.501126 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db2c47-558d-4fa1-b264-b06d46579b9d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:41.523179 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db2c47-558d-4fa1-b264-b06d46579b9d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:41.533119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2b152-fd90-4938-b6a2-74af75448565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:41.534721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2b152-fd90-4938-b6a2-74af75448565 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:41.536439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2b152-fd90-4938-b6a2-74af75448565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:41.537204 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2b152-fd90-4938-b6a2-74af75448565 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:41.539890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2b152-fd90-4938-b6a2-74af75448565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:41.544604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2b152-fd90-4938-b6a2-74af75448565 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:41.578773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1110/2213] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:46:41.579424 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25376078-5062-4f61-b2f6-e87ac33813d8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:41.604176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25376078-5062-4f61-b2f6-e87ac33813d8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:41.629425 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164c9c2-e547-49c7-8be8-17edbcd2481b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:41.631391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164c9c2-e547-49c7-8be8-17edbcd2481b admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:41.631992 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164c9c2-e547-49c7-8be8-17edbcd2481b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:41.632873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164c9c2-e547-49c7-8be8-17edbcd2481b admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:41.633987 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164c9c2-e547-49c7-8be8-17edbcd2481b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:41.637868 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164c9c2-e547-49c7-8be8-17edbcd2481b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:41.677260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1104/2214] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:46:41.682170 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2b152-fd90-4938-b6a2-74af75448565 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:41.706933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2b152-fd90-4938-b6a2-74af75448565 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:41.714665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73216e25-f6fa-417e-9d63-5a117186654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:41.715898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73216e25-f6fa-417e-9d63-5a117186654e admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:41.715898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73216e25-f6fa-417e-9d63-5a117186654e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:41.715898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73216e25-f6fa-417e-9d63-5a117186654e admin admin] PATH_INFO: `/v3/projects/1c7bf87ab9734866b53b47258dd908ec/users/66d2dfb3c1194122888a8bbe6b9605ec/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:41.753622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73216e25-f6fa-417e-9d63-5a117186654e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7bf87ab9734866b53b47258dd908ec, user_id=66d2dfb3c1194122888a8bbe6b9605ec, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:41.757365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73216e25-f6fa-417e-9d63-5a117186654e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:41.796783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1111/2215] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:46:41 2022] PUT /identity/v3/projects/1c7bf87ab9734866b53b47258dd908ec/users/66d2dfb3c1194122888a8bbe6b9605ec/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:46:41.807719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14380a93-0567-4b0e-a82d-6b37149dbcdf None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:41.808431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14380a93-0567-4b0e-a82d-6b37149dbcdf None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:41.808431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14380a93-0567-4b0e-a82d-6b37149dbcdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:41.875595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-14380a93-0567-4b0e-a82d-6b37149dbcdf None None] MFA Rules not processed for user `66d2dfb3c1194122888a8bbe6b9605ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:41.876869 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-14380a93-0567-4b0e-a82d-6b37149dbcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:42.090262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1105/2216] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:46:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:46:42.102178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73216e25-f6fa-417e-9d63-5a117186654e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:42.105018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-73216e25-f6fa-417e-9d63-5a117186654e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:42.250312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73216e25-f6fa-417e-9d63-5a117186654e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:42.255868 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858e807-663b-462d-bcfb-8478e53e1e38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:42.256980 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858e807-663b-462d-bcfb-8478e53e1e38 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:42.257445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858e807-663b-462d-bcfb-8478e53e1e38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:42.257752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858e807-663b-462d-bcfb-8478e53e1e38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:42.277759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858e807-663b-462d-bcfb-8478e53e1e38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:42.279616 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858e807-663b-462d-bcfb-8478e53e1e38 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:42.292198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1112/2217] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:46:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:46:48.567890 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14380a93-0567-4b0e-a82d-6b37149dbcdf None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:48.570187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-14380a93-0567-4b0e-a82d-6b37149dbcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:48.660899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14380a93-0567-4b0e-a82d-6b37149dbcdf None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:48.665275 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4e68d-e4c4-4793-ab99-7d9acc74992e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:48.666904 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4e68d-e4c4-4793-ab99-7d9acc74992e service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:48.667209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4e68d-e4c4-4793-ab99-7d9acc74992e service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:48.667527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4e68d-e4c4-4793-ab99-7d9acc74992e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:48.669910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fcf4e68d-e4c4-4793-ab99-7d9acc74992e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:48.702570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:48.771478 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4e68d-e4c4-4793-ab99-7d9acc74992e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:48.773441 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4e68d-e4c4-4793-ab99-7d9acc74992e service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:48.785701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1106/2218] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:46:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:46:51.416590 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2c820-b3ad-46a0-bbb3-d46cc54a1de1 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:51.417147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2c820-b3ad-46a0-bbb3-d46cc54a1de1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:51.417710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2c820-b3ad-46a0-bbb3-d46cc54a1de1 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:51.420371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1113/2219] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 3 10:46:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:46:51.429142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae7894-9841-49b3-b7fe-da992a1c15c7 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:51.429870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae7894-9841-49b3-b7fe-da992a1c15c7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:51.432022 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae7894-9841-49b3-b7fe-da992a1c15c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:51.458853 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2aae7894-9841-49b3-b7fe-da992a1c15c7 None None] MFA Rules not processed for user `e66f25cc329f418eba6bd637bf3e8a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:51.460290 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2aae7894-9841-49b3-b7fe-da992a1c15c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:51.573614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1107/2220] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 3 10:46:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:46:51.593175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2c820-b3ad-46a0-bbb3-d46cc54a1de1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:51.596551 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7ee2c820-b3ad-46a0-bbb3-d46cc54a1de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:51.621792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:46:51.711282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2c820-b3ad-46a0-bbb3-d46cc54a1de1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:51.715951 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fb97c-4de1-41bc-82f3-b425858ae87d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '300fb5e1c5c546808b935871673e3d0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:51.716978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fb97c-4de1-41bc-82f3-b425858ae87d service glance] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:51.717244 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fb97c-4de1-41bc-82f3-b425858ae87d service glance] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:51.717755 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fb97c-4de1-41bc-82f3-b425858ae87d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:51.741083 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fb97c-4de1-41bc-82f3-b425858ae87d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:51.744248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fb97c-4de1-41bc-82f3-b425858ae87d service glance] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:51.769885 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1114/2221] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:46:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:46:51.859488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035bc4d-1b32-4eb3-a026-5d2e4bb48089 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:51.860258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035bc4d-1b32-4eb3-a026-5d2e4bb48089 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:51.860738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035bc4d-1b32-4eb3-a026-5d2e4bb48089 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:51.863947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1108/2222] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 3 10:46:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:46:51.873072 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1b5bb-9362-4e40-b45f-b2630cd5ace2 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:51.873498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1b5bb-9362-4e40-b45f-b2630cd5ace2 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:51.873857 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1b5bb-9362-4e40-b45f-b2630cd5ace2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:51.899720 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-65a1b5bb-9362-4e40-b45f-b2630cd5ace2 None None] MFA Rules not processed for user `1c211cc19a2045bd9383d31c7e81ceac`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:51.901090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-65a1b5bb-9362-4e40-b45f-b2630cd5ace2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:51.910547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1115/2223] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 3 10:46:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:46:51.926829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035bc4d-1b32-4eb3-a026-5d2e4bb48089 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:51.945148 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035bc4d-1b32-4eb3-a026-5d2e4bb48089 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:51.950784 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60404052-f6e1-4d8a-a602-f689eeff8f28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:51.951428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60404052-f6e1-4d8a-a602-f689eeff8f28 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:51.951710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60404052-f6e1-4d8a-a602-f689eeff8f28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:51.952749 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60404052-f6e1-4d8a-a602-f689eeff8f28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:51.961996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60404052-f6e1-4d8a-a602-f689eeff8f28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:51.963648 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60404052-f6e1-4d8a-a602-f689eeff8f28 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:51.977723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1109/2224] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:46:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:46:52.113255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1098e7b-3955-4eea-999e-0e30b28ea21b None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:52.113825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1098e7b-3955-4eea-999e-0e30b28ea21b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:52.114259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1098e7b-3955-4eea-999e-0e30b28ea21b None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:52.116331 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1116/2225] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 3 10:46:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:46:52.131994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712a70c-2e31-4477-a790-10da33eb2d23 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:52.132535 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712a70c-2e31-4477-a790-10da33eb2d23 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:52.132967 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712a70c-2e31-4477-a790-10da33eb2d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:52.157564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b712a70c-2e31-4477-a790-10da33eb2d23 None None] MFA Rules not processed for user `e66f25cc329f418eba6bd637bf3e8a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:46:52.159457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b712a70c-2e31-4477-a790-10da33eb2d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:46:52.178015 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1110/2226] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 3 10:46:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:46:52.196899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1098e7b-3955-4eea-999e-0e30b28ea21b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:46:52.210549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1098e7b-3955-4eea-999e-0e30b28ea21b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:46:52.214757 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bca8e-23a5-4831-b79a-2649ef683381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:46:52.215345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bca8e-23a5-4831-b79a-2649ef683381 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:46:52.215883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bca8e-23a5-4831-b79a-2649ef683381 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:46:52.216193 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bca8e-23a5-4831-b79a-2649ef683381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:46:52.232620 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205bca8e-23a5-4831-b79a-2649ef683381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:46:52.234573 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205bca8e-23a5-4831-b79a-2649ef683381 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:46:52.256647 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1117/2227] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:46:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:47:00.201380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8933aa9-b9de-45b5-a3cc-d8ba302db4b1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:00.202225 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8933aa9-b9de-45b5-a3cc-d8ba302db4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:00.202225 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8933aa9-b9de-45b5-a3cc-d8ba302db4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:00.226001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b8933aa9-b9de-45b5-a3cc-d8ba302db4b1 None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:47:00.228143 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b8933aa9-b9de-45b5-a3cc-d8ba302db4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:47:00.345283 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1111/2228] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 3 10:47:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:47:00.366605 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bca8e-23a5-4831-b79a-2649ef683381 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:47:00.367239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-205bca8e-23a5-4831-b79a-2649ef683381 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:47:00.481934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bca8e-23a5-4831-b79a-2649ef683381 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:47:00.492648 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d9d04-934d-4f1e-baae-dc85ef7b13a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:47:00.493576 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d9d04-934d-4f1e-baae-dc85ef7b13a3 service swift] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:00.494159 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d9d04-934d-4f1e-baae-dc85ef7b13a3 service swift] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:00.494752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d9d04-934d-4f1e-baae-dc85ef7b13a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:00.510296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d9d04-934d-4f1e-baae-dc85ef7b13a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:47:00.513552 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d9d04-934d-4f1e-baae-dc85ef7b13a3 service swift] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:47:00.530978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1118/2229] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:47:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 10:47:10.328928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=85 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:47:10.523027 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=61 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:47:26.943245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7d33d-c60b-47b1-ae9e-55af0afeb0b1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:26.943844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7d33d-c60b-47b1-ae9e-55af0afeb0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:26.943844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7d33d-c60b-47b1-ae9e-55af0afeb0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:26.990053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-14d7d33d-c60b-47b1-ae9e-55af0afeb0b1 None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:47:26.991548 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-14d7d33d-c60b-47b1-ae9e-55af0afeb0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:47:27.002980 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1112/2230] 10.222.0.47 () {58 vars in 967 bytes} [Wed Aug 3 10:47:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:47:27.021136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d9d04-934d-4f1e-baae-dc85ef7b13a3 service swift] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:47:27.034170 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d9d04-934d-4f1e-baae-dc85ef7b13a3 service swift] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:47:27.051075 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176113d-249d-4ee6-9411-611dbe737e33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:47:27.051879 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176113d-249d-4ee6-9411-611dbe737e33 service swift] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:27.052296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176113d-249d-4ee6-9411-611dbe737e33 service swift] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:27.052689 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176113d-249d-4ee6-9411-611dbe737e33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:27.069932 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176113d-249d-4ee6-9411-611dbe737e33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:47:27.071745 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176113d-249d-4ee6-9411-611dbe737e33 service swift] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:47:27.084490 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1119/2231] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:47:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 10:47:36.976282 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:47:37.078041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:47:45.068013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77408d-a693-4129-9a7a-a5a0a9c3f370 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:45.068476 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77408d-a693-4129-9a7a-a5a0a9c3f370 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:45.068700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77408d-a693-4129-9a7a-a5a0a9c3f370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:45.089269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5f77408d-a693-4129-9a7a-a5a0a9c3f370 None None] MFA Rules not processed for user `956aff831fa04d29ba5cffb88e73f4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:47:45.090496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5f77408d-a693-4129-9a7a-a5a0a9c3f370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:47:45.151850 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1113/2232] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:47:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:47:45.721587 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176113d-249d-4ee6-9411-611dbe737e33 service swift] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:47:45.823680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176113d-249d-4ee6-9411-611dbe737e33 service swift] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:47:45.829842 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a2406-3bb0-48a5-88c1-80432022841b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:47:45.830547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a2406-3bb0-48a5-88c1-80432022841b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:45.830822 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a2406-3bb0-48a5-88c1-80432022841b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:45.831275 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a2406-3bb0-48a5-88c1-80432022841b admin admin] PATH_INFO: `/v3/users/66d2dfb3c1194122888a8bbe6b9605ec` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:45.833582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a2406-3bb0-48a5-88c1-80432022841b admin admin] RBAC: Authorizing `identity:delete_user(user_id=66d2dfb3c1194122888a8bbe6b9605ec)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:47:45.836029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a2406-3bb0-48a5-88c1-80432022841b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:47:45.907976 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-608a2406-3bb0-48a5-88c1-80432022841b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66d2dfb3c1194122888a8bbe6b9605ec'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:47:45.927065 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-608a2406-3bb0-48a5-88c1-80432022841b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66d2dfb3c1194122888a8bbe6b9605ec'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:47:45.927836 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-608a2406-3bb0-48a5-88c1-80432022841b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66d2dfb3c1194122888a8bbe6b9605ec'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:47:45.937533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-608a2406-3bb0-48a5-88c1-80432022841b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66d2dfb3c1194122888a8bbe6b9605ec'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:47:45.946227 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-608a2406-3bb0-48a5-88c1-80432022841b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66d2dfb3c1194122888a8bbe6b9605ec'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:47:45.955656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1120/2233] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:47:45 2022] DELETE /identity/v3/users/66d2dfb3c1194122888a8bbe6b9605ec => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:47:45.960898 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6ee28-551d-46f4-bb4c-20bc6974abad None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:45.961204 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6ee28-551d-46f4-bb4c-20bc6974abad None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:45.961519 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6ee28-551d-46f4-bb4c-20bc6974abad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:45.983639 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-11f6ee28-551d-46f4-bb4c-20bc6974abad None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:47:45.984897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-11f6ee28-551d-46f4-bb4c-20bc6974abad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:47:46.057362 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1114/2234] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:47:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:47:46.068624 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a2406-3bb0-48a5-88c1-80432022841b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:47:46.070438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-608a2406-3bb0-48a5-88c1-80432022841b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:47:46.158467 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a2406-3bb0-48a5-88c1-80432022841b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:47:46.163887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f488a2-c915-42ae-ba91-4c3ae73f9d4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:47:46.164628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f488a2-c915-42ae-ba91-4c3ae73f9d4b service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:46.164912 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f488a2-c915-42ae-ba91-4c3ae73f9d4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:46.165235 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f488a2-c915-42ae-ba91-4c3ae73f9d4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:46.185701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f488a2-c915-42ae-ba91-4c3ae73f9d4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:47:46.188980 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f488a2-c915-42ae-ba91-4c3ae73f9d4b service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:47:46.205230 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1121/2235] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:47:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:47:46.562861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6ee28-551d-46f4-bb4c-20bc6974abad None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:47:46.564574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-11f6ee28-551d-46f4-bb4c-20bc6974abad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:47:46.578731 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6ee28-551d-46f4-bb4c-20bc6974abad None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:47:46.588026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddaa810-1f81-4394-82e5-18dcbc61adce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:47:46.589701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaa810-1f81-4394-82e5-18dcbc61adce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:46.590486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaa810-1f81-4394-82e5-18dcbc61adce admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:46.590746 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaa810-1f81-4394-82e5-18dcbc61adce admin admin] PATH_INFO: `/v3/projects/1c7bf87ab9734866b53b47258dd908ec` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:46.593212 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddaa810-1f81-4394-82e5-18dcbc61adce admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7bf87ab9734866b53b47258dd908ec)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:47:46.597221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddaa810-1f81-4394-82e5-18dcbc61adce admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:47:46.670233 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dddaa810-1f81-4394-82e5-18dcbc61adce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7bf87ab9734866b53b47258dd908ec', 'request_id': 'req-dddaa810-1f81-4394-82e5-18dcbc61adce'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:47:46.677894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dddaa810-1f81-4394-82e5-18dcbc61adce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7bf87ab9734866b53b47258dd908ec', 'request_id': 'req-dddaa810-1f81-4394-82e5-18dcbc61adce'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:47:46.692860 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dddaa810-1f81-4394-82e5-18dcbc61adce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7bf87ab9734866b53b47258dd908ec', 'request_id': 'req-dddaa810-1f81-4394-82e5-18dcbc61adce'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:47:46.703504 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dddaa810-1f81-4394-82e5-18dcbc61adce admin admin] The token cache is being invalidate because project 1c7bf87ab9734866b53b47258dd908ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:47:46.703798 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dddaa810-1f81-4394-82e5-18dcbc61adce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:47:46.704433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dddaa810-1f81-4394-82e5-18dcbc61adce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:47:46.706117 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1115/2236] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:47:46 2022] DELETE /identity/v3/projects/1c7bf87ab9734866b53b47258dd908ec => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:47:46.710992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf2e07-b9aa-47b3-a07a-0cf21204913d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:46.711283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf2e07-b9aa-47b3-a07a-0cf21204913d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:46.711563 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf2e07-b9aa-47b3-a07a-0cf21204913d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:46.730589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-91bf2e07-b9aa-47b3-a07a-0cf21204913d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:47:46.731791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-91bf2e07-b9aa-47b3-a07a-0cf21204913d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:47:46.753263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:47:46.803807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1122/2237] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:47:46.811530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32e3ce-0811-40f2-80f6-3dab050792b9 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:46.811943 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32e3ce-0811-40f2-80f6-3dab050792b9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:46.812451 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32e3ce-0811-40f2-80f6-3dab050792b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:46.834063 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0e32e3ce-0811-40f2-80f6-3dab050792b9 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:47:46.835404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0e32e3ce-0811-40f2-80f6-3dab050792b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:47:46.846124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1116/2238] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:47:46.850688 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf2e07-b9aa-47b3-a07a-0cf21204913d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:47:46.861256 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf2e07-b9aa-47b3-a07a-0cf21204913d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:47:46.867158 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8f99f-ebf2-4da1-aa70-97b1ad87925a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:47:46.867893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8f99f-ebf2-4da1-aa70-97b1ad87925a admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:46.868187 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8f99f-ebf2-4da1-aa70-97b1ad87925a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:46.868463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8f99f-ebf2-4da1-aa70-97b1ad87925a admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:46.869268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8f99f-ebf2-4da1-aa70-97b1ad87925a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:47:46.869686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8f99f-ebf2-4da1-aa70-97b1ad87925a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:47:46.871435 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8f99f-ebf2-4da1-aa70-97b1ad87925a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:47:46.884614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1123/2239] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:47:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:47:46.889534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32e3ce-0811-40f2-80f6-3dab050792b9 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:47:46.901607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32e3ce-0811-40f2-80f6-3dab050792b9 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:47:46.906304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fb7f0-d7a2-47c7-8bdf-82c99ddc0d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:47:46.906938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fb7f0-d7a2-47c7-8bdf-82c99ddc0d1c admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:46.907236 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fb7f0-d7a2-47c7-8bdf-82c99ddc0d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:46.907495 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fb7f0-d7a2-47c7-8bdf-82c99ddc0d1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:46.908172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573fb7f0-d7a2-47c7-8bdf-82c99ddc0d1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:47:46.910158 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573fb7f0-d7a2-47c7-8bdf-82c99ddc0d1c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:47:46.934609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:47:46.942488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1117/2240] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:47:46 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:47:46.946542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8f99f-ebf2-4da1-aa70-97b1ad87925a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:47:46.958597 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8f99f-ebf2-4da1-aa70-97b1ad87925a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:47:47.022518 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5295b-fb8f-4e79-ab8a-35e2e4e49223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:47:47.022765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5295b-fb8f-4e79-ab8a-35e2e4e49223 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:47.022920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5295b-fb8f-4e79-ab8a-35e2e4e49223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:47.023143 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5295b-fb8f-4e79-ab8a-35e2e4e49223 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:47.023825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5295b-fb8f-4e79-ab8a-35e2e4e49223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:47:47.025786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5295b-fb8f-4e79-ab8a-35e2e4e49223 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:47:47.051352 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1124/2241] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:47:46 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:47:47.056103 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fb7f0-d7a2-47c7-8bdf-82c99ddc0d1c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:47:47.068289 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fb7f0-d7a2-47c7-8bdf-82c99ddc0d1c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:47:47.074485 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ed753-bf61-4478-92b8-7ede423b17e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:47:47.075287 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ed753-bf61-4478-92b8-7ede423b17e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:47.075634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ed753-bf61-4478-92b8-7ede423b17e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:47.075983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ed753-bf61-4478-92b8-7ede423b17e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:47.076724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ed753-bf61-4478-92b8-7ede423b17e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:47:47.078622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ed753-bf61-4478-92b8-7ede423b17e4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:47:47.093544 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1118/2242] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:47:47.097810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5295b-fb8f-4e79-ab8a-35e2e4e49223 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:47:47.108979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5295b-fb8f-4e79-ab8a-35e2e4e49223 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:47:47.116300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d2bda-8fa6-430a-b3fd-e7502d68c9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:47:47.116933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d2bda-8fa6-430a-b3fd-e7502d68c9dd admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:47.117219 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d2bda-8fa6-430a-b3fd-e7502d68c9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:47.117659 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d2bda-8fa6-430a-b3fd-e7502d68c9dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:47.118283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d2bda-8fa6-430a-b3fd-e7502d68c9dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:47:47.119832 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d2bda-8fa6-430a-b3fd-e7502d68c9dd admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:47:47.132330 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1125/2243] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:47:47.137143 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ed753-bf61-4478-92b8-7ede423b17e4 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:47:47.147625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ed753-bf61-4478-92b8-7ede423b17e4 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:47:47.153050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214278e-78f5-41f1-a4ba-3dd5dcdb38fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:47:47.153936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214278e-78f5-41f1-a4ba-3dd5dcdb38fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:47.154257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214278e-78f5-41f1-a4ba-3dd5dcdb38fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:47.154488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214278e-78f5-41f1-a4ba-3dd5dcdb38fe admin admin] PATH_INFO: `/v3/projects/79ec0038f90e431dbe8ac58bf109e259/users/ea2b889e606f453e8fd5bc0710072fa6/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:47.173908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214278e-78f5-41f1-a4ba-3dd5dcdb38fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=79ec0038f90e431dbe8ac58bf109e259, user_id=ea2b889e606f453e8fd5bc0710072fa6, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:47:47.178403 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214278e-78f5-41f1-a4ba-3dd5dcdb38fe admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:47:47.201769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1119/2244] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:47:47 2022] PUT /identity/v3/projects/79ec0038f90e431dbe8ac58bf109e259/users/ea2b889e606f453e8fd5bc0710072fa6/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:47:47.215326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15ed4a-9bfd-4c26-9739-899b8f1c9445 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:47.215611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15ed4a-9bfd-4c26-9739-899b8f1c9445 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:47.215889 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15ed4a-9bfd-4c26-9739-899b8f1c9445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:47.266638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-df15ed4a-9bfd-4c26-9739-899b8f1c9445 None None] MFA Rules not processed for user `ea2b889e606f453e8fd5bc0710072fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:47:47.268083 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-df15ed4a-9bfd-4c26-9739-899b8f1c9445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:47:47.354804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1126/2245] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:47:47.370695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214278e-78f5-41f1-a4ba-3dd5dcdb38fe admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:47:47.373504 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6214278e-78f5-41f1-a4ba-3dd5dcdb38fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:47:47.456205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214278e-78f5-41f1-a4ba-3dd5dcdb38fe admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:47:47.460991 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef39351-142b-4d1d-b354-644a0ed27b5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:47:47.461899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef39351-142b-4d1d-b354-644a0ed27b5e service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:47.462280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef39351-142b-4d1d-b354-644a0ed27b5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:47.462657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef39351-142b-4d1d-b354-644a0ed27b5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:47.472846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef39351-142b-4d1d-b354-644a0ed27b5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:47:47.475153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef39351-142b-4d1d-b354-644a0ed27b5e service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:47:47.484609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1120/2246] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:47:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:47:48.884052 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466ebac-eaf8-40b6-acb5-03249e08f680 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:47:48.884627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466ebac-eaf8-40b6-acb5-03249e08f680 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:47:48.884627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466ebac-eaf8-40b6-acb5-03249e08f680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:47:48.904951 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4466ebac-eaf8-40b6-acb5-03249e08f680 None None] MFA Rules not processed for user `ea2b889e606f453e8fd5bc0710072fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:47:48.906316 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4466ebac-eaf8-40b6-acb5-03249e08f680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:47:48.914036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1127/2247] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:47:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:47:57.480204 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:47:58.899132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:48:00.104407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc222313-82a1-4808-b115-b9ed05dd4c8a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:00.105079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc222313-82a1-4808-b115-b9ed05dd4c8a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:00.105079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc222313-82a1-4808-b115-b9ed05dd4c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:00.124779 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-dc222313-82a1-4808-b115-b9ed05dd4c8a None None] MFA Rules not processed for user `92485e486d53488eabf12105d1048e13`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:48:00.126060 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-dc222313-82a1-4808-b115-b9ed05dd4c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:00.183268 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1121/2248] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:48:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:48:10.176341 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=25 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:48:11.161448 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f24ce7-34bb-43f8-bcf9-2996e945efc2 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:11.161928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f24ce7-34bb-43f8-bcf9-2996e945efc2 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:11.162021 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f24ce7-34bb-43f8-bcf9-2996e945efc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:11.182245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f1f24ce7-34bb-43f8-bcf9-2996e945efc2 None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:48:11.183491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f1f24ce7-34bb-43f8-bcf9-2996e945efc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:11.244865 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1128/2249] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 3 10:48:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:48:11.267044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc222313-82a1-4808-b115-b9ed05dd4c8a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:48:11.268513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-dc222313-82a1-4808-b115-b9ed05dd4c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:11.337122 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc222313-82a1-4808-b115-b9ed05dd4c8a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:48:11.342120 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8456ff2-05e0-4e24-bb8a-6e4f54093605 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:48:11.342743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8456ff2-05e0-4e24-bb8a-6e4f54093605 service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:11.343027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8456ff2-05e0-4e24-bb8a-6e4f54093605 service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:11.343306 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8456ff2-05e0-4e24-bb8a-6e4f54093605 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:11.351647 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8456ff2-05e0-4e24-bb8a-6e4f54093605 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:48:11.353251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8456ff2-05e0-4e24-bb8a-6e4f54093605 service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:48:11.362365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1122/2250] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:48:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 10:48:11.987460 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6577444-c17d-4fe0-afc6-bc85ce3a8e5b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:11.987743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6577444-c17d-4fe0-afc6-bc85ce3a8e5b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:11.988050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6577444-c17d-4fe0-afc6-bc85ce3a8e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:12.009266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b6577444-c17d-4fe0-afc6-bc85ce3a8e5b None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:48:12.010326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b6577444-c17d-4fe0-afc6-bc85ce3a8e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:12.018770 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1129/2251] 10.222.0.47 () {58 vars in 967 bytes} [Wed Aug 3 10:48:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:48:12.032356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8456ff2-05e0-4e24-bb8a-6e4f54093605 service swift] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:48:12.043814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8456ff2-05e0-4e24-bb8a-6e4f54093605 service swift] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:48:12.049587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8409bb-f87c-494a-b0b0-b5a0472ea602 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:48:12.050215 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8409bb-f87c-494a-b0b0-b5a0472ea602 service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:12.050500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8409bb-f87c-494a-b0b0-b5a0472ea602 service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:12.050845 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8409bb-f87c-494a-b0b0-b5a0472ea602 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:12.060142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8409bb-f87c-494a-b0b0-b5a0472ea602 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:48:12.061748 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8409bb-f87c-494a-b0b0-b5a0472ea602 service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:48:12.070594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1123/2252] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:48:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 10:48:12.295131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0d27a-790a-4f09-b15c-ca567649c673 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:12.295623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0d27a-790a-4f09-b15c-ca567649c673 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:12.295716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0d27a-790a-4f09-b15c-ca567649c673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:12.315480 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-37e0d27a-790a-4f09-b15c-ca567649c673 None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:48:12.316709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-37e0d27a-790a-4f09-b15c-ca567649c673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:12.324684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1130/2253] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 3 10:48:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:48:12.336609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8409bb-f87c-494a-b0b0-b5a0472ea602 service swift] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:48:12.348304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8409bb-f87c-494a-b0b0-b5a0472ea602 service swift] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:48:12.353905 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb100c-a186-4d17-a7df-bf5ebff45c29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:48:12.354574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb100c-a186-4d17-a7df-bf5ebff45c29 service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:12.354860 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb100c-a186-4d17-a7df-bf5ebff45c29 service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:12.355194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb100c-a186-4d17-a7df-bf5ebff45c29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:12.364165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb100c-a186-4d17-a7df-bf5ebff45c29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:48:12.366372 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb100c-a186-4d17-a7df-bf5ebff45c29 service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:48:12.375207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1124/2254] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:48:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 10:48:12.716763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fed8e-9822-4c44-8be8-06d508fd1160 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:12.717050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fed8e-9822-4c44-8be8-06d508fd1160 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:12.717329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fed8e-9822-4c44-8be8-06d508fd1160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:12.737333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8d4fed8e-9822-4c44-8be8-06d508fd1160 None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:48:12.738607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8d4fed8e-9822-4c44-8be8-06d508fd1160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:12.747246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1131/2255] 10.222.0.47 () {58 vars in 967 bytes} [Wed Aug 3 10:48:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:48:12.760816 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb100c-a186-4d17-a7df-bf5ebff45c29 service swift] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:48:12.774195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb100c-a186-4d17-a7df-bf5ebff45c29 service swift] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:48:12.791779 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e92679-1e8f-4624-9d0a-aa01bafa8497 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:48:12.792429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e92679-1e8f-4624-9d0a-aa01bafa8497 service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:12.792789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e92679-1e8f-4624-9d0a-aa01bafa8497 service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:12.792974 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e92679-1e8f-4624-9d0a-aa01bafa8497 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:12.804524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e92679-1e8f-4624-9d0a-aa01bafa8497 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:48:12.806526 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e92679-1e8f-4624-9d0a-aa01bafa8497 service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:48:12.818306 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1125/2256] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:48:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 10:48:15.063911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fed8e-9822-4c44-8be8-06d508fd1160 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:48:15.154440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fed8e-9822-4c44-8be8-06d508fd1160 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:48:15.159557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c70686-8703-4d69-9ef5-63922ee2b173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:48:15.160237 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c70686-8703-4d69-9ef5-63922ee2b173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:15.160522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c70686-8703-4d69-9ef5-63922ee2b173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:15.160800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c70686-8703-4d69-9ef5-63922ee2b173 admin admin] PATH_INFO: `/v3/users/ea2b889e606f453e8fd5bc0710072fa6` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:15.162176 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c70686-8703-4d69-9ef5-63922ee2b173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea2b889e606f453e8fd5bc0710072fa6)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:48:15.164155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c70686-8703-4d69-9ef5-63922ee2b173 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:48:15.241699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d5c70686-8703-4d69-9ef5-63922ee2b173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea2b889e606f453e8fd5bc0710072fa6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:48:15.256159 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d5c70686-8703-4d69-9ef5-63922ee2b173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea2b889e606f453e8fd5bc0710072fa6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:48:15.257074 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d5c70686-8703-4d69-9ef5-63922ee2b173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea2b889e606f453e8fd5bc0710072fa6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:48:15.267022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d5c70686-8703-4d69-9ef5-63922ee2b173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea2b889e606f453e8fd5bc0710072fa6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:48:15.277121 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d5c70686-8703-4d69-9ef5-63922ee2b173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea2b889e606f453e8fd5bc0710072fa6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:48:15.284937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1132/2257] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:48:15 2022] DELETE /identity/v3/users/ea2b889e606f453e8fd5bc0710072fa6 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:48:15.290303 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea8beb-7765-4094-bf7b-8c0da868a512 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:15.290848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea8beb-7765-4094-bf7b-8c0da868a512 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:15.290948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea8beb-7765-4094-bf7b-8c0da868a512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:15.311829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a6ea8beb-7765-4094-bf7b-8c0da868a512 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:48:15.313268 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a6ea8beb-7765-4094-bf7b-8c0da868a512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:15.386773 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1126/2258] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:48:15.399714 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c70686-8703-4d69-9ef5-63922ee2b173 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:48:15.401232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d5c70686-8703-4d69-9ef5-63922ee2b173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:15.409000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:48:15.488462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c70686-8703-4d69-9ef5-63922ee2b173 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:48:15.492526 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8ccd8-c0a1-4e50-a45a-178098be46c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:48:15.493166 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8ccd8-c0a1-4e50-a45a-178098be46c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:15.493453 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8ccd8-c0a1-4e50-a45a-178098be46c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:15.493764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8ccd8-c0a1-4e50-a45a-178098be46c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:15.504117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8ccd8-c0a1-4e50-a45a-178098be46c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:48:15.506138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8ccd8-c0a1-4e50-a45a-178098be46c7 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:48:15.519680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1133/2259] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:48:15.781559 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea8beb-7765-4094-bf7b-8c0da868a512 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:48:15.783128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a6ea8beb-7765-4094-bf7b-8c0da868a512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:15.798563 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea8beb-7765-4094-bf7b-8c0da868a512 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:48:15.804145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcd740-fcc8-475b-9e49-84f066d30e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:48:15.804833 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcd740-fcc8-475b-9e49-84f066d30e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:15.805111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcd740-fcc8-475b-9e49-84f066d30e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:15.806101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcd740-fcc8-475b-9e49-84f066d30e4d admin admin] PATH_INFO: `/v3/projects/79ec0038f90e431dbe8ac58bf109e259` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:15.808037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcd740-fcc8-475b-9e49-84f066d30e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=79ec0038f90e431dbe8ac58bf109e259)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:48:15.809412 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcd740-fcc8-475b-9e49-84f066d30e4d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:48:15.868772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:48:15.871713 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8adcd740-fcc8-475b-9e49-84f066d30e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79ec0038f90e431dbe8ac58bf109e259', 'request_id': 'req-8adcd740-fcc8-475b-9e49-84f066d30e4d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:48:15.879284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8adcd740-fcc8-475b-9e49-84f066d30e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79ec0038f90e431dbe8ac58bf109e259', 'request_id': 'req-8adcd740-fcc8-475b-9e49-84f066d30e4d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:48:15.889736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8adcd740-fcc8-475b-9e49-84f066d30e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79ec0038f90e431dbe8ac58bf109e259', 'request_id': 'req-8adcd740-fcc8-475b-9e49-84f066d30e4d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:48:15.902502 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8adcd740-fcc8-475b-9e49-84f066d30e4d admin admin] The token cache is being invalidate because project 79ec0038f90e431dbe8ac58bf109e259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:48:15.902786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8adcd740-fcc8-475b-9e49-84f066d30e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:48:15.903446 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8adcd740-fcc8-475b-9e49-84f066d30e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:48:15.904983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1127/2260] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:48:15 2022] DELETE /identity/v3/projects/79ec0038f90e431dbe8ac58bf109e259 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:48:25.512360 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=24 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:48:25.899273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:48:38.417361 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e3a8a-4352-492e-b3e1-a89c6178d6b6 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:38.418390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e3a8a-4352-492e-b3e1-a89c6178d6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:38.418956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e3a8a-4352-492e-b3e1-a89c6178d6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:38.435505 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcd740-fcc8-475b-9e49-84f066d30e4d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:48:38.437974 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8adcd740-fcc8-475b-9e49-84f066d30e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:38.442080 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-709e3a8a-4352-492e-b3e1-a89c6178d6b6 None None] MFA Rules not processed for user `5744cccc5a224263b71c052dc783e103`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:48:38.443748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-709e3a8a-4352-492e-b3e1-a89c6178d6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:38.538476 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1134/2261] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:48:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:48:38.538709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcd740-fcc8-475b-9e49-84f066d30e4d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:48:38.541742 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba73a38-ee2b-4966-b106-b3e3e3e958df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:48:38.542558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba73a38-ee2b-4966-b106-b3e3e3e958df service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:38.542940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba73a38-ee2b-4966-b106-b3e3e3e958df service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:38.543317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba73a38-ee2b-4966-b106-b3e3e3e958df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:38.545361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7ba73a38-ee2b-4966-b106-b3e3e3e958df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:38.639693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba73a38-ee2b-4966-b106-b3e3e3e958df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:48:38.644798 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba73a38-ee2b-4966-b106-b3e3e3e958df service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:48:38.657641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1128/2262] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:48:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:48:38.883111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319586c-b10b-4bad-8fee-d09bd8f03ced None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:38.883403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319586c-b10b-4bad-8fee-d09bd8f03ced None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:38.883683 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319586c-b10b-4bad-8fee-d09bd8f03ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:38.902651 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3319586c-b10b-4bad-8fee-d09bd8f03ced None None] MFA Rules not processed for user `ebdb81686d804eb3864b97481643fc2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:48:38.903882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3319586c-b10b-4bad-8fee-d09bd8f03ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:38.973838 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1135/2263] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:48:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:48:39.648271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba73a38-ee2b-4966-b106-b3e3e3e958df service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:48:39.660319 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba73a38-ee2b-4966-b106-b3e3e3e958df service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:48:39.673965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545fcda-84b5-4a31-9f22-4aee8d830baf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:48:39.674494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545fcda-84b5-4a31-9f22-4aee8d830baf service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:39.674958 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545fcda-84b5-4a31-9f22-4aee8d830baf service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:39.675610 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545fcda-84b5-4a31-9f22-4aee8d830baf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:39.677588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4545fcda-84b5-4a31-9f22-4aee8d830baf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:39.708814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545fcda-84b5-4a31-9f22-4aee8d830baf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:48:39.711080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545fcda-84b5-4a31-9f22-4aee8d830baf service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:48:39.733451 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1129/2264] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:48:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:48:41.343274 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20579456-6f52-4bf0-a586-8754825dd226 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:41.343773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20579456-6f52-4bf0-a586-8754825dd226 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:41.343914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20579456-6f52-4bf0-a586-8754825dd226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:41.362764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-20579456-6f52-4bf0-a586-8754825dd226 None None] MFA Rules not processed for user `a5e714aad25f4fda8acf767ad7e2490b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:48:41.363956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-20579456-6f52-4bf0-a586-8754825dd226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:41.419920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1136/2265] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:48:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:48:49.725593 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=60 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:48:50.114933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545fcda-84b5-4a31-9f22-4aee8d830baf service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:48:50.116753 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4545fcda-84b5-4a31-9f22-4aee8d830baf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:50.130649 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545fcda-84b5-4a31-9f22-4aee8d830baf service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:48:50.136327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240bf64b-546d-44b1-85a5-c0bfe2647a62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:48:50.136992 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bf64b-546d-44b1-85a5-c0bfe2647a62 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:50.137538 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bf64b-546d-44b1-85a5-c0bfe2647a62 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:50.137836 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bf64b-546d-44b1-85a5-c0bfe2647a62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:50.140124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-240bf64b-546d-44b1-85a5-c0bfe2647a62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:50.154829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240bf64b-546d-44b1-85a5-c0bfe2647a62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:48:50.157846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240bf64b-546d-44b1-85a5-c0bfe2647a62 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:48:50.169428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1130/2266] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:48:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:48:51.412950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=75 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:48:55.159596 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20579456-6f52-4bf0-a586-8754825dd226 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:48:55.164070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-20579456-6f52-4bf0-a586-8754825dd226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:55.293955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20579456-6f52-4bf0-a586-8754825dd226 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:48:55.297669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104b3f0-ebc6-4037-92fd-34cd9056241b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:48:55.298306 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104b3f0-ebc6-4037-92fd-34cd9056241b service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:48:55.298592 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104b3f0-ebc6-4037-92fd-34cd9056241b service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:48:55.298869 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104b3f0-ebc6-4037-92fd-34cd9056241b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:48:55.300707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3104b3f0-ebc6-4037-92fd-34cd9056241b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:48:55.378806 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104b3f0-ebc6-4037-92fd-34cd9056241b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:48:55.380475 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104b3f0-ebc6-4037-92fd-34cd9056241b service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:48:55.391471 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1137/2267] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:48:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:49:00.163713 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:49:05.388075 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=48 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:49:06.842794 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240bf64b-546d-44b1-85a5-c0bfe2647a62 service nova] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:06.844372 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-240bf64b-546d-44b1-85a5-c0bfe2647a62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:06.916935 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240bf64b-546d-44b1-85a5-c0bfe2647a62 service nova] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:06.921537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b303f3-866b-481d-9a1a-c93132a534e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:06.922202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b303f3-866b-481d-9a1a-c93132a534e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:06.922617 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b303f3-866b-481d-9a1a-c93132a534e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:06.922896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b303f3-866b-481d-9a1a-c93132a534e0 admin admin] PATH_INFO: `/v3/users/92485e486d53488eabf12105d1048e13` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:06.924101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b303f3-866b-481d-9a1a-c93132a534e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92485e486d53488eabf12105d1048e13)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:06.926087 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b303f3-866b-481d-9a1a-c93132a534e0 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:06.988024 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d1b303f3-866b-481d-9a1a-c93132a534e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92485e486d53488eabf12105d1048e13'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:06.998117 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d1b303f3-866b-481d-9a1a-c93132a534e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92485e486d53488eabf12105d1048e13'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:07.006911 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d1b303f3-866b-481d-9a1a-c93132a534e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92485e486d53488eabf12105d1048e13'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:07.007673 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d1b303f3-866b-481d-9a1a-c93132a534e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92485e486d53488eabf12105d1048e13'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:07.008568 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d1b303f3-866b-481d-9a1a-c93132a534e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92485e486d53488eabf12105d1048e13'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:07.029363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1131/2268] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:49:06 2022] DELETE /identity/v3/users/92485e486d53488eabf12105d1048e13 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:49:07.233457 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104b3f0-ebc6-4037-92fd-34cd9056241b service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:07.247611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104b3f0-ebc6-4037-92fd-34cd9056241b service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:07.328485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c772f3-5258-443c-9e9f-7978fbc880fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:07.329157 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c772f3-5258-443c-9e9f-7978fbc880fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:07.329446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c772f3-5258-443c-9e9f-7978fbc880fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:07.329757 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c772f3-5258-443c-9e9f-7978fbc880fb admin admin] PATH_INFO: `/v3/projects/f9d4304db6d342e4ada925b6784e6d0d` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:07.330995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c772f3-5258-443c-9e9f-7978fbc880fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9d4304db6d342e4ada925b6784e6d0d)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:07.333134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c772f3-5258-443c-9e9f-7978fbc880fb admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:07.392936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-75c772f3-5258-443c-9e9f-7978fbc880fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9d4304db6d342e4ada925b6784e6d0d', 'request_id': 'req-75c772f3-5258-443c-9e9f-7978fbc880fb'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:07.401238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-75c772f3-5258-443c-9e9f-7978fbc880fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9d4304db6d342e4ada925b6784e6d0d', 'request_id': 'req-75c772f3-5258-443c-9e9f-7978fbc880fb'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:07.408391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-75c772f3-5258-443c-9e9f-7978fbc880fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9d4304db6d342e4ada925b6784e6d0d', 'request_id': 'req-75c772f3-5258-443c-9e9f-7978fbc880fb'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:07.420431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-75c772f3-5258-443c-9e9f-7978fbc880fb admin admin] The token cache is being invalidate because project f9d4304db6d342e4ada925b6784e6d0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:49:07.420692 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-75c772f3-5258-443c-9e9f-7978fbc880fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:07.421581 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-75c772f3-5258-443c-9e9f-7978fbc880fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:07.423323 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1138/2269] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:49:07 2022] DELETE /identity/v3/projects/f9d4304db6d342e4ada925b6784e6d0d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:49:07.428321 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcce599-5827-4721-be35-a74599623c8e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:07.428607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcce599-5827-4721-be35-a74599623c8e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:07.429434 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcce599-5827-4721-be35-a74599623c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:07.450469 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-9dcce599-5827-4721-be35-a74599623c8e None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:49:07.451750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9dcce599-5827-4721-be35-a74599623c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:07.526336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1132/2270] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:49:07.534844 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6f2db-3aae-4322-8751-02263897b4b8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:07.535461 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6f2db-3aae-4322-8751-02263897b4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:07.535938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6f2db-3aae-4322-8751-02263897b4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:07.569828 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4ce6f2db-3aae-4322-8751-02263897b4b8 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:49:07.571078 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4ce6f2db-3aae-4322-8751-02263897b4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:07.580938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1139/2271] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:49:07.585437 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcce599-5827-4721-be35-a74599623c8e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:07.595576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcce599-5827-4721-be35-a74599623c8e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:07.600276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3c573-aabc-45f5-bd26-efc5d93ae6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:07.601492 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3c573-aabc-45f5-bd26-efc5d93ae6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:07.601780 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3c573-aabc-45f5-bd26-efc5d93ae6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:07.602050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3c573-aabc-45f5-bd26-efc5d93ae6e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:07.602757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3c573-aabc-45f5-bd26-efc5d93ae6e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:49:07.603060 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3c573-aabc-45f5-bd26-efc5d93ae6e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:07.604684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3c573-aabc-45f5-bd26-efc5d93ae6e8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:07.623086 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1133/2272] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:49:07.628259 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6f2db-3aae-4322-8751-02263897b4b8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:07.639216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6f2db-3aae-4322-8751-02263897b4b8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:07.644975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07861117-db91-4841-82af-c5878bda24e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:07.645872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07861117-db91-4841-82af-c5878bda24e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:07.646210 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07861117-db91-4841-82af-c5878bda24e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:07.646544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07861117-db91-4841-82af-c5878bda24e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:07.647305 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07861117-db91-4841-82af-c5878bda24e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:07.649730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07861117-db91-4841-82af-c5878bda24e0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:07.687247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1140/2273] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:49:07 2022] POST /identity/v3/projects => generated 399 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:49:07.691765 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3c573-aabc-45f5-bd26-efc5d93ae6e8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:07.709082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3c573-aabc-45f5-bd26-efc5d93ae6e8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:07.713453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:49:07.842133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11b73f-8fed-4e66-bf1a-42acb431506f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:07.843056 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11b73f-8fed-4e66-bf1a-42acb431506f admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:07.843601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11b73f-8fed-4e66-bf1a-42acb431506f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:07.843601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11b73f-8fed-4e66-bf1a-42acb431506f admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:07.844864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11b73f-8fed-4e66-bf1a-42acb431506f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:07.846854 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11b73f-8fed-4e66-bf1a-42acb431506f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:07.877183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1134/2274] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:49:07 2022] POST /identity/v3/users => generated 300 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:49:07.881439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07861117-db91-4841-82af-c5878bda24e0 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:07.895194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07861117-db91-4841-82af-c5878bda24e0 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:07.901653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814e04f-c273-4469-9e8f-d0bb8f0df8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:07.902298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814e04f-c273-4469-9e8f-d0bb8f0df8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:07.902572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814e04f-c273-4469-9e8f-d0bb8f0df8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:07.902861 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814e04f-c273-4469-9e8f-d0bb8f0df8d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:07.903566 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814e04f-c273-4469-9e8f-d0bb8f0df8d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:07.905399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814e04f-c273-4469-9e8f-d0bb8f0df8d4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:07.918216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1141/2275] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:49:07.922802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11b73f-8fed-4e66-bf1a-42acb431506f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:07.937922 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11b73f-8fed-4e66-bf1a-42acb431506f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:07.944126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a83c2c-6685-4b4c-84fd-b6ec288d08da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:07.945258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a83c2c-6685-4b4c-84fd-b6ec288d08da admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:07.945795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a83c2c-6685-4b4c-84fd-b6ec288d08da admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:07.946285 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a83c2c-6685-4b4c-84fd-b6ec288d08da admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:07.947603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a83c2c-6685-4b4c-84fd-b6ec288d08da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:07.950747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a83c2c-6685-4b4c-84fd-b6ec288d08da admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:07.964995 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1135/2276] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:49:07.969828 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814e04f-c273-4469-9e8f-d0bb8f0df8d4 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:07.983500 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814e04f-c273-4469-9e8f-d0bb8f0df8d4 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:07.991972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb90c4a-b79d-4f1e-8497-f406bf8628dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:07.993284 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb90c4a-b79d-4f1e-8497-f406bf8628dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:07.993917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb90c4a-b79d-4f1e-8497-f406bf8628dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:07.994439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb90c4a-b79d-4f1e-8497-f406bf8628dc admin admin] PATH_INFO: `/v3/projects/406a455363fc4538b29df5b22d09048a/users/cd11cd1298d844c08ffa8a5ef37d1add/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:08.014269 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb90c4a-b79d-4f1e-8497-f406bf8628dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=406a455363fc4538b29df5b22d09048a, user_id=cd11cd1298d844c08ffa8a5ef37d1add, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:08.017072 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb90c4a-b79d-4f1e-8497-f406bf8628dc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:08.035365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1142/2277] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:49:07 2022] PUT /identity/v3/projects/406a455363fc4538b29df5b22d09048a/users/cd11cd1298d844c08ffa8a5ef37d1add/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:49:08.047034 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a561497-5b69-42e5-acea-1eac9b0a0424 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:08.047148 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a561497-5b69-42e5-acea-1eac9b0a0424 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:08.047472 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a561497-5b69-42e5-acea-1eac9b0a0424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:08.078455 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0a561497-5b69-42e5-acea-1eac9b0a0424 None None] MFA Rules not processed for user `cd11cd1298d844c08ffa8a5ef37d1add`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:49:08.079876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0a561497-5b69-42e5-acea-1eac9b0a0424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:08.150873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1136/2278] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:49:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:49:08.162784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb90c4a-b79d-4f1e-8497-f406bf8628dc admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:08.165200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8eb90c4a-b79d-4f1e-8497-f406bf8628dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:08.200562 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:49:08.242958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb90c4a-b79d-4f1e-8497-f406bf8628dc admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:08.250367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc32d69-a1dc-49ab-8a96-e1a4e6603afe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:08.251553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc32d69-a1dc-49ab-8a96-e1a4e6603afe service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:08.252137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc32d69-a1dc-49ab-8a96-e1a4e6603afe service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:08.252734 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc32d69-a1dc-49ab-8a96-e1a4e6603afe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:08.270984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc32d69-a1dc-49ab-8a96-e1a4e6603afe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:08.274019 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc32d69-a1dc-49ab-8a96-e1a4e6603afe service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:08.310191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1143/2279] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:49:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:49:10.581822 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930aede-6c6c-497e-ac41-43cdd8c1f29a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:10.582401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930aede-6c6c-497e-ac41-43cdd8c1f29a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:10.582401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930aede-6c6c-497e-ac41-43cdd8c1f29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:10.602892 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a930aede-6c6c-497e-ac41-43cdd8c1f29a None None] MFA Rules not processed for user `956aff831fa04d29ba5cffb88e73f4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:49:10.604192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a930aede-6c6c-497e-ac41-43cdd8c1f29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:10.669358 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1137/2280] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:49:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:49:10.975327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc32d69-a1dc-49ab-8a96-e1a4e6603afe service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:10.976753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7dc32d69-a1dc-49ab-8a96-e1a4e6603afe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:11.033206 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc32d69-a1dc-49ab-8a96-e1a4e6603afe service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:11.036806 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45bb38-9a95-4346-bb77-8d7b7d61e6d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:11.037420 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45bb38-9a95-4346-bb77-8d7b7d61e6d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:11.037739 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45bb38-9a95-4346-bb77-8d7b7d61e6d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:11.038018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45bb38-9a95-4346-bb77-8d7b7d61e6d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:11.039772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5a45bb38-9a95-4346-bb77-8d7b7d61e6d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:11.049022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45bb38-9a95-4346-bb77-8d7b7d61e6d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:11.050462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45bb38-9a95-4346-bb77-8d7b7d61e6d9 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:11.058781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1144/2281] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:49:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:49:11.388985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930aede-6c6c-497e-ac41-43cdd8c1f29a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:11.390475 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a930aede-6c6c-497e-ac41-43cdd8c1f29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:11.416885 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:49:11.471311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930aede-6c6c-497e-ac41-43cdd8c1f29a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:11.477026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb12e9-cb78-498d-8156-0727d78a3aaa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2c5536eddd346d692822a2f3befb48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:11.477680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb12e9-cb78-498d-8156-0727d78a3aaa service placement] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:11.477962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb12e9-cb78-498d-8156-0727d78a3aaa service placement] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:11.478237 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb12e9-cb78-498d-8156-0727d78a3aaa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:11.480006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-dbdb12e9-cb78-498d-8156-0727d78a3aaa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:11.492968 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb12e9-cb78-498d-8156-0727d78a3aaa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:11.494715 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb12e9-cb78-498d-8156-0727d78a3aaa service placement] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:11.504940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1138/2282] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:49:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:49:15.592551 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45bb38-9a95-4346-bb77-8d7b7d61e6d9 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:15.661819 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45bb38-9a95-4346-bb77-8d7b7d61e6d9 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:15.666928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcfd31-3db0-4e02-ae1a-78b51c209814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:15.667570 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcfd31-3db0-4e02-ae1a-78b51c209814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:15.667851 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcfd31-3db0-4e02-ae1a-78b51c209814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:15.668126 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcfd31-3db0-4e02-ae1a-78b51c209814 admin admin] PATH_INFO: `/v3/users/cd11cd1298d844c08ffa8a5ef37d1add` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:15.669299 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcfd31-3db0-4e02-ae1a-78b51c209814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd11cd1298d844c08ffa8a5ef37d1add)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:15.671316 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcfd31-3db0-4e02-ae1a-78b51c209814 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:15.716519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-12dcfd31-3db0-4e02-ae1a-78b51c209814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd11cd1298d844c08ffa8a5ef37d1add'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:15.729793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-12dcfd31-3db0-4e02-ae1a-78b51c209814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd11cd1298d844c08ffa8a5ef37d1add'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:15.730493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-12dcfd31-3db0-4e02-ae1a-78b51c209814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd11cd1298d844c08ffa8a5ef37d1add'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:15.739151 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-12dcfd31-3db0-4e02-ae1a-78b51c209814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd11cd1298d844c08ffa8a5ef37d1add'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:15.741340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:49:15.749057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-12dcfd31-3db0-4e02-ae1a-78b51c209814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd11cd1298d844c08ffa8a5ef37d1add'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:15.755202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1145/2283] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:49:15 2022] DELETE /identity/v3/users/cd11cd1298d844c08ffa8a5ef37d1add => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:49:15.760448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123a7a7-aaf8-49f0-a04b-7378029ed7c7 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:15.760927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123a7a7-aaf8-49f0-a04b-7378029ed7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:15.761082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123a7a7-aaf8-49f0-a04b-7378029ed7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:15.781768 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a123a7a7-aaf8-49f0-a04b-7378029ed7c7 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:49:15.782968 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a123a7a7-aaf8-49f0-a04b-7378029ed7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:15.841565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1139/2284] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:49:15.851501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcfd31-3db0-4e02-ae1a-78b51c209814 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:15.852920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-12dcfd31-3db0-4e02-ae1a-78b51c209814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:15.911657 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcfd31-3db0-4e02-ae1a-78b51c209814 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:15.915354 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa031a58-6cfb-4439-a572-6e55cf16c0b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:15.915974 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa031a58-6cfb-4439-a572-6e55cf16c0b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:15.916254 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa031a58-6cfb-4439-a572-6e55cf16c0b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:15.916527 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa031a58-6cfb-4439-a572-6e55cf16c0b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:15.926207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa031a58-6cfb-4439-a572-6e55cf16c0b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:15.927626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa031a58-6cfb-4439-a572-6e55cf16c0b7 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:15.937441 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1146/2285] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:49:16.717907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123a7a7-aaf8-49f0-a04b-7378029ed7c7 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:16.719613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a123a7a7-aaf8-49f0-a04b-7378029ed7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:16.733095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123a7a7-aaf8-49f0-a04b-7378029ed7c7 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:16.738514 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89617b30-b884-4f72-ae51-45e4ae18c47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:16.739188 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89617b30-b884-4f72-ae51-45e4ae18c47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:16.739737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89617b30-b884-4f72-ae51-45e4ae18c47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:16.747054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89617b30-b884-4f72-ae51-45e4ae18c47f admin admin] PATH_INFO: `/v3/projects/406a455363fc4538b29df5b22d09048a` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:16.747054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89617b30-b884-4f72-ae51-45e4ae18c47f admin admin] RBAC: Authorizing `identity:delete_project(project_id=406a455363fc4538b29df5b22d09048a)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:16.747054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89617b30-b884-4f72-ae51-45e4ae18c47f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:16.814569 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-89617b30-b884-4f72-ae51-45e4ae18c47f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '406a455363fc4538b29df5b22d09048a', 'request_id': 'req-89617b30-b884-4f72-ae51-45e4ae18c47f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:16.821681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-89617b30-b884-4f72-ae51-45e4ae18c47f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '406a455363fc4538b29df5b22d09048a', 'request_id': 'req-89617b30-b884-4f72-ae51-45e4ae18c47f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:16.832034 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-89617b30-b884-4f72-ae51-45e4ae18c47f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '406a455363fc4538b29df5b22d09048a', 'request_id': 'req-89617b30-b884-4f72-ae51-45e4ae18c47f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:16.845299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-89617b30-b884-4f72-ae51-45e4ae18c47f admin admin] The token cache is being invalidate because project 406a455363fc4538b29df5b22d09048a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:49:16.845626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-89617b30-b884-4f72-ae51-45e4ae18c47f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:16.846320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-89617b30-b884-4f72-ae51-45e4ae18c47f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:16.847885 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1140/2286] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:49:16 2022] DELETE /identity/v3/projects/406a455363fc4538b29df5b22d09048a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:49:22.637615 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa031a58-6cfb-4439-a572-6e55cf16c0b7 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:22.639099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-aa031a58-6cfb-4439-a572-6e55cf16c0b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:22.702119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa031a58-6cfb-4439-a572-6e55cf16c0b7 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:22.750462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69454a98-219c-4556-8ad5-6ac6de7d3c8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:22.751083 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69454a98-219c-4556-8ad5-6ac6de7d3c8e service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:22.751357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69454a98-219c-4556-8ad5-6ac6de7d3c8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:22.751628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69454a98-219c-4556-8ad5-6ac6de7d3c8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:22.753417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-69454a98-219c-4556-8ad5-6ac6de7d3c8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:22.811124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69454a98-219c-4556-8ad5-6ac6de7d3c8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:22.812563 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69454a98-219c-4556-8ad5-6ac6de7d3c8e service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:22.823015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1147/2287] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:49:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:49:26.841772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:49:29.338935 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89617b30-b884-4f72-ae51-45e4ae18c47f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:29.340525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-89617b30-b884-4f72-ae51-45e4ae18c47f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:29.404914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89617b30-b884-4f72-ae51-45e4ae18c47f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:29.408656 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea973a-f73d-4443-87ae-45d08150b1d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:29.409275 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea973a-f73d-4443-87ae-45d08150b1d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:29.409588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea973a-f73d-4443-87ae-45d08150b1d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:29.409862 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea973a-f73d-4443-87ae-45d08150b1d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:29.411601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f1ea973a-f73d-4443-87ae-45d08150b1d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:29.421748 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea973a-f73d-4443-87ae-45d08150b1d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:29.423153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea973a-f73d-4443-87ae-45d08150b1d2 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:29.431582 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1141/2288] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:49:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:49:32.817403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:49:34.776083 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f4782-780f-42ae-8d72-8fc4bedcb947 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:34.776965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f4782-780f-42ae-8d72-8fc4bedcb947 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:34.776965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f4782-780f-42ae-8d72-8fc4bedcb947 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:34.778855 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1148/2289] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 3 10:49:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:49:34.786194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dba35c-500d-47df-a15b-c268f917f44f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:34.788559 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dba35c-500d-47df-a15b-c268f917f44f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:34.789020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dba35c-500d-47df-a15b-c268f917f44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:34.812067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a1dba35c-500d-47df-a15b-c268f917f44f None None] MFA Rules not processed for user `1c211cc19a2045bd9383d31c7e81ceac`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:49:34.813425 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a1dba35c-500d-47df-a15b-c268f917f44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:34.821297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1142/2290] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 3 10:49:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:49:34.836555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f4782-780f-42ae-8d72-8fc4bedcb947 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:34.852338 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f4782-780f-42ae-8d72-8fc4bedcb947 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:34.856569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c724a-347c-43d8-ba2e-846509efcff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:34.857334 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c724a-347c-43d8-ba2e-846509efcff6 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:34.857776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c724a-347c-43d8-ba2e-846509efcff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:34.858212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c724a-347c-43d8-ba2e-846509efcff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:34.871588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c724a-347c-43d8-ba2e-846509efcff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:34.873389 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c724a-347c-43d8-ba2e-846509efcff6 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:34.883950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1149/2291] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:49:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:49:35.227511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ab055-04ec-4be8-92d2-f3b77b95063d None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:35.227797 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ab055-04ec-4be8-92d2-f3b77b95063d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:35.228073 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ab055-04ec-4be8-92d2-f3b77b95063d None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:35.229496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1143/2292] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 3 10:49:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:49:35.235826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15e432-33bb-493d-aaa9-d45eb1c6f7c9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:35.236167 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15e432-33bb-493d-aaa9-d45eb1c6f7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:35.236449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15e432-33bb-493d-aaa9-d45eb1c6f7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:35.258675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9c15e432-33bb-493d-aaa9-d45eb1c6f7c9 None None] MFA Rules not processed for user `e66f25cc329f418eba6bd637bf3e8a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:49:35.260228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9c15e432-33bb-493d-aaa9-d45eb1c6f7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:35.347855 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1150/2293] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 3 10:49:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:49:35.359791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ab055-04ec-4be8-92d2-f3b77b95063d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:35.362013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-339ab055-04ec-4be8-92d2-f3b77b95063d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:35.382105 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ab055-04ec-4be8-92d2-f3b77b95063d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:35.387054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a2bf1-0efa-4941-8885-f57abd71a39e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:35.388069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a2bf1-0efa-4941-8885-f57abd71a39e service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:35.388769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a2bf1-0efa-4941-8885-f57abd71a39e service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:35.389208 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a2bf1-0efa-4941-8885-f57abd71a39e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:35.402951 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a2bf1-0efa-4941-8885-f57abd71a39e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:35.404995 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a2bf1-0efa-4941-8885-f57abd71a39e service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:35.418072 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1144/2294] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:49:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:49:38.583133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15e432-33bb-493d-aaa9-d45eb1c6f7c9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:38.585721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9c15e432-33bb-493d-aaa9-d45eb1c6f7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:38.613937 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15e432-33bb-493d-aaa9-d45eb1c6f7c9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:38.624253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53176be-44e9-4de2-aca7-4dc4e8784069 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:38.625810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53176be-44e9-4de2-aca7-4dc4e8784069 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:38.628132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53176be-44e9-4de2-aca7-4dc4e8784069 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:38.628132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53176be-44e9-4de2-aca7-4dc4e8784069 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:38.630986 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e53176be-44e9-4de2-aca7-4dc4e8784069 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:38.651594 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53176be-44e9-4de2-aca7-4dc4e8784069 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:38.653304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53176be-44e9-4de2-aca7-4dc4e8784069 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:38.668478 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1151/2295] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:49:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:49:45.412439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=35 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:49:45.568848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a2bf1-0efa-4941-8885-f57abd71a39e service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:45.578607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a2bf1-0efa-4941-8885-f57abd71a39e service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:45.582044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d8536-b167-42d0-a684-04f452d5455d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:45.582667 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d8536-b167-42d0-a684-04f452d5455d service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:45.582948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d8536-b167-42d0-a684-04f452d5455d service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:45.583218 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d8536-b167-42d0-a684-04f452d5455d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:45.584983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-dd7d8536-b167-42d0-a684-04f452d5455d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:45.657912 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d8536-b167-42d0-a684-04f452d5455d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:45.659359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d8536-b167-42d0-a684-04f452d5455d service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:45.672002 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1145/2296] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:49:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:49:47.263394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53176be-44e9-4de2-aca7-4dc4e8784069 service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:47.275279 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53176be-44e9-4de2-aca7-4dc4e8784069 service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:47.281176 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d5627-55b4-4059-a82f-7bdc2ef307f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:47.281828 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d5627-55b4-4059-a82f-7bdc2ef307f2 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:47.282114 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d5627-55b4-4059-a82f-7bdc2ef307f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:47.282393 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d5627-55b4-4059-a82f-7bdc2ef307f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:47.284257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-055d5627-55b4-4059-a82f-7bdc2ef307f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:47.354117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d5627-55b4-4059-a82f-7bdc2ef307f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:47.356027 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d5627-55b4-4059-a82f-7bdc2ef307f2 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:47.369297 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1152/2297] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:49:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:49:49.536960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467282a-4307-4341-b493-9f3250b36f71 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:49.538152 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467282a-4307-4341-b493-9f3250b36f71 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:49.539264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467282a-4307-4341-b493-9f3250b36f71 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:49.542553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1146/2298] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 3 10:49:49 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 10:49:49.578041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3c177-24a6-4084-9e7a-47e480533254 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:49.578499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3c177-24a6-4084-9e7a-47e480533254 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:49.578676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3c177-24a6-4084-9e7a-47e480533254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:49.602328 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-73b3c177-24a6-4084-9e7a-47e480533254 None None] MFA Rules not processed for user `f2c5536eddd346d692822a2f3befb48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:49:49.603491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-73b3c177-24a6-4084-9e7a-47e480533254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:49.626577 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467282a-4307-4341-b493-9f3250b36f71 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:49.628747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8467282a-4307-4341-b493-9f3250b36f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:49.664655 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467282a-4307-4341-b493-9f3250b36f71 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:49.677812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd98e8-cec8-4fd4-9fa6-d98b42184934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:49.678881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd98e8-cec8-4fd4-9fa6-d98b42184934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:49.681010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd98e8-cec8-4fd4-9fa6-d98b42184934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:49.681676 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd98e8-cec8-4fd4-9fa6-d98b42184934 admin admin] PATH_INFO: `/v3/users/a5e714aad25f4fda8acf767ad7e2490b` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:49.694817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd98e8-cec8-4fd4-9fa6-d98b42184934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5e714aad25f4fda8acf767ad7e2490b)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:49.699193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd98e8-cec8-4fd4-9fa6-d98b42184934 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:49.732507 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:49:49.746529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1153/2299] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 3 10:49:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:49:49.769450 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5cd98e8-cec8-4fd4-9fa6-d98b42184934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5e714aad25f4fda8acf767ad7e2490b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:49.780280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5cd98e8-cec8-4fd4-9fa6-d98b42184934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5e714aad25f4fda8acf767ad7e2490b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:49.793940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5cd98e8-cec8-4fd4-9fa6-d98b42184934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5e714aad25f4fda8acf767ad7e2490b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:49.795067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5cd98e8-cec8-4fd4-9fa6-d98b42184934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5e714aad25f4fda8acf767ad7e2490b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:49.796151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5cd98e8-cec8-4fd4-9fa6-d98b42184934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5e714aad25f4fda8acf767ad7e2490b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:49.808723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3c177-24a6-4084-9e7a-47e480533254 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:49.811003 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-73b3c177-24a6-4084-9e7a-47e480533254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:49.834599 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1147/2300] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:49:49 2022] DELETE /identity/v3/users/a5e714aad25f4fda8acf767ad7e2490b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:49:49.835291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3c177-24a6-4084-9e7a-47e480533254 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:49.975950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0dd95-8464-4ec8-a387-227eeac85f96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2c5536eddd346d692822a2f3befb48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:49.977541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0dd95-8464-4ec8-a387-227eeac85f96 service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:49.978216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0dd95-8464-4ec8-a387-227eeac85f96 service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:49.978922 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0dd95-8464-4ec8-a387-227eeac85f96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:49.982391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2eb0dd95-8464-4ec8-a387-227eeac85f96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:50.009368 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0dd95-8464-4ec8-a387-227eeac85f96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:50.015593 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0dd95-8464-4ec8-a387-227eeac85f96 service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:50.031904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1154/2301] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:49:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:49:50.118358 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd98e8-cec8-4fd4-9fa6-d98b42184934 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:50.120843 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f5cd98e8-cec8-4fd4-9fa6-d98b42184934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:50.202128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd98e8-cec8-4fd4-9fa6-d98b42184934 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:50.206749 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563213eb-e5f5-4225-84b8-89d551ca6bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:50.207402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563213eb-e5f5-4225-84b8-89d551ca6bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:50.207686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563213eb-e5f5-4225-84b8-89d551ca6bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:50.207961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563213eb-e5f5-4225-84b8-89d551ca6bb9 admin admin] PATH_INFO: `/v3/projects/ca8789771e8a4505866424b6c7748291` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:50.209119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563213eb-e5f5-4225-84b8-89d551ca6bb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca8789771e8a4505866424b6c7748291)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:50.211104 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563213eb-e5f5-4225-84b8-89d551ca6bb9 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:50.219618 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:49:50.262840 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-563213eb-e5f5-4225-84b8-89d551ca6bb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca8789771e8a4505866424b6c7748291', 'request_id': 'req-563213eb-e5f5-4225-84b8-89d551ca6bb9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:50.269367 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-563213eb-e5f5-4225-84b8-89d551ca6bb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca8789771e8a4505866424b6c7748291', 'request_id': 'req-563213eb-e5f5-4225-84b8-89d551ca6bb9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:50.277901 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-563213eb-e5f5-4225-84b8-89d551ca6bb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca8789771e8a4505866424b6c7748291', 'request_id': 'req-563213eb-e5f5-4225-84b8-89d551ca6bb9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:50.290125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-563213eb-e5f5-4225-84b8-89d551ca6bb9 admin admin] The token cache is being invalidate because project ca8789771e8a4505866424b6c7748291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:49:50.290421 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-563213eb-e5f5-4225-84b8-89d551ca6bb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:50.291135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-563213eb-e5f5-4225-84b8-89d551ca6bb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:49:50.292658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1148/2302] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:49:50 2022] DELETE /identity/v3/projects/ca8789771e8a4505866424b6c7748291 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:49:50.297680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a618b16-0db6-4317-9266-073abada6aa4 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:50.298057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a618b16-0db6-4317-9266-073abada6aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:50.298343 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a618b16-0db6-4317-9266-073abada6aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:50.319447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5a618b16-0db6-4317-9266-073abada6aa4 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:49:50.320681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5a618b16-0db6-4317-9266-073abada6aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:50.382286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1155/2303] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:49:50.390299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e85af-1650-4e01-b9fb-a7485cc74e14 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:50.390566 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e85af-1650-4e01-b9fb-a7485cc74e14 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:50.390845 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e85af-1650-4e01-b9fb-a7485cc74e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:50.411550 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-575e85af-1650-4e01-b9fb-a7485cc74e14 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:49:50.412931 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-575e85af-1650-4e01-b9fb-a7485cc74e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:50.422224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1149/2304] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:49:50.427701 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a618b16-0db6-4317-9266-073abada6aa4 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:50.439411 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a618b16-0db6-4317-9266-073abada6aa4 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:50.444316 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f9692-aa4a-4029-97e0-42fcb87084d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:50.444955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f9692-aa4a-4029-97e0-42fcb87084d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:50.445268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f9692-aa4a-4029-97e0-42fcb87084d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:50.445697 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f9692-aa4a-4029-97e0-42fcb87084d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:50.446287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f9692-aa4a-4029-97e0-42fcb87084d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:49:50.446646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f9692-aa4a-4029-97e0-42fcb87084d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:50.448295 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f9692-aa4a-4029-97e0-42fcb87084d3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:50.467132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1156/2305] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:49:50.469228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e85af-1650-4e01-b9fb-a7485cc74e14 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:50.482881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e85af-1650-4e01-b9fb-a7485cc74e14 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:50.490314 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3523d-d089-42ca-bf02-4bce36bfc8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:50.491058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3523d-d089-42ca-bf02-4bce36bfc8f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:50.491421 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3523d-d089-42ca-bf02-4bce36bfc8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:50.491704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3523d-d089-42ca-bf02-4bce36bfc8f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:50.492984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3523d-d089-42ca-bf02-4bce36bfc8f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:50.494954 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3523d-d089-42ca-bf02-4bce36bfc8f8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:50.532042 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1150/2306] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:49:50 2022] POST /identity/v3/projects => generated 363 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:49:50.538423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f9692-aa4a-4029-97e0-42fcb87084d3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:50.549538 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f9692-aa4a-4029-97e0-42fcb87084d3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:50.612333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1f84d-daec-4419-8fc7-20a6c4987a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:50.612998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1f84d-daec-4419-8fc7-20a6c4987a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:50.613232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1f84d-daec-4419-8fc7-20a6c4987a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:50.613819 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1f84d-daec-4419-8fc7-20a6c4987a5b admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:50.614491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1f84d-daec-4419-8fc7-20a6c4987a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:50.616431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1f84d-daec-4419-8fc7-20a6c4987a5b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:50.637116 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1157/2307] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:49:50 2022] POST /identity/v3/users => generated 282 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:49:50.641316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3523d-d089-42ca-bf02-4bce36bfc8f8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:50.653143 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3523d-d089-42ca-bf02-4bce36bfc8f8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:50.657838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9402d-0657-4388-a684-11a2cd836de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:50.658617 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9402d-0657-4388-a684-11a2cd836de9 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:50.658909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9402d-0657-4388-a684-11a2cd836de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:50.659182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9402d-0657-4388-a684-11a2cd836de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:50.659879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9402d-0657-4388-a684-11a2cd836de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:50.661537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9402d-0657-4388-a684-11a2cd836de9 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:50.672710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1151/2308] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:49:50.677110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1f84d-daec-4419-8fc7-20a6c4987a5b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:50.687883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1f84d-daec-4419-8fc7-20a6c4987a5b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:50.693242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730d7f6-53b6-4501-9499-20ba14e72a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:50.693909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730d7f6-53b6-4501-9499-20ba14e72a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:50.694196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730d7f6-53b6-4501-9499-20ba14e72a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:50.694476 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730d7f6-53b6-4501-9499-20ba14e72a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:50.695221 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730d7f6-53b6-4501-9499-20ba14e72a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:50.696819 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730d7f6-53b6-4501-9499-20ba14e72a8f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:50.698265 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:49:50.712786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1158/2309] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:49:50.714089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9402d-0657-4388-a684-11a2cd836de9 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:50.724503 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9402d-0657-4388-a684-11a2cd836de9 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:50.729861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec19dc7-9f7d-4126-b9de-4ded2217881c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:50.730550 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec19dc7-9f7d-4126-b9de-4ded2217881c admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:50.730859 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec19dc7-9f7d-4126-b9de-4ded2217881c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:50.731136 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec19dc7-9f7d-4126-b9de-4ded2217881c admin admin] PATH_INFO: `/v3/projects/7e02d017fdd54381bbc2d3793182e667/users/170faab1c46045fba061143dfe1c0872/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:50.748082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec19dc7-9f7d-4126-b9de-4ded2217881c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e02d017fdd54381bbc2d3793182e667, user_id=170faab1c46045fba061143dfe1c0872, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:50.750828 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec19dc7-9f7d-4126-b9de-4ded2217881c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:50.764889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1152/2310] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:49:50 2022] PUT /identity/v3/projects/7e02d017fdd54381bbc2d3793182e667/users/170faab1c46045fba061143dfe1c0872/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:49:50.777172 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d23200-f804-42aa-a6c9-049ee28116d1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:50.777484 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d23200-f804-42aa-a6c9-049ee28116d1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:50.777794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d23200-f804-42aa-a6c9-049ee28116d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:50.812275 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-97d23200-f804-42aa-a6c9-049ee28116d1 None None] MFA Rules not processed for user `170faab1c46045fba061143dfe1c0872`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:49:50.813555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-97d23200-f804-42aa-a6c9-049ee28116d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:50.882583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1159/2311] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:49:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:49:50.887454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec19dc7-9f7d-4126-b9de-4ded2217881c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:50.954073 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec19dc7-9f7d-4126-b9de-4ded2217881c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:50.958933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d2d49-498e-4695-b385-2a938d38d82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:50.959574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d2d49-498e-4695-b385-2a938d38d82b admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:50.959859 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d2d49-498e-4695-b385-2a938d38d82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:50.960144 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d2d49-498e-4695-b385-2a938d38d82b admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:50.961420 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d2d49-498e-4695-b385-2a938d38d82b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:50.963361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d2d49-498e-4695-b385-2a938d38d82b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:50.996823 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1153/2312] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:49:50 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:49:51.001851 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d23200-f804-42aa-a6c9-049ee28116d1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:51.013662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d23200-f804-42aa-a6c9-049ee28116d1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:51.089526 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac811d5-a9b4-4b09-9e51-46f927bb3557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:51.090482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac811d5-a9b4-4b09-9e51-46f927bb3557 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:51.090771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac811d5-a9b4-4b09-9e51-46f927bb3557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:51.091054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac811d5-a9b4-4b09-9e51-46f927bb3557 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:51.091754 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac811d5-a9b4-4b09-9e51-46f927bb3557 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:51.093952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac811d5-a9b4-4b09-9e51-46f927bb3557 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:51.133451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1160/2313] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:49:51 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:49:51.139106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d2d49-498e-4695-b385-2a938d38d82b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:51.153715 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d2d49-498e-4695-b385-2a938d38d82b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:51.159145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56e214-8ab5-4b99-a8d4-082e730131b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:51.159906 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56e214-8ab5-4b99-a8d4-082e730131b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:51.160244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56e214-8ab5-4b99-a8d4-082e730131b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:51.160530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56e214-8ab5-4b99-a8d4-082e730131b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:51.161342 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f56e214-8ab5-4b99-a8d4-082e730131b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:51.163897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f56e214-8ab5-4b99-a8d4-082e730131b3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:51.177142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1154/2314] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:49:51.183101 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac811d5-a9b4-4b09-9e51-46f927bb3557 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:51.199608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac811d5-a9b4-4b09-9e51-46f927bb3557 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:51.208609 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4890bd-6e10-472b-9fdc-641bf0f557f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:51.210235 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4890bd-6e10-472b-9fdc-641bf0f557f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:51.211255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4890bd-6e10-472b-9fdc-641bf0f557f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:51.211733 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4890bd-6e10-472b-9fdc-641bf0f557f6 admin admin] PATH_INFO: `/v3/projects/4c25a16ad5454e1b805f08036c76eb4f/users/8ead692343d24d709bde96d0be60de10/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:51.237558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4890bd-6e10-472b-9fdc-641bf0f557f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c25a16ad5454e1b805f08036c76eb4f, user_id=8ead692343d24d709bde96d0be60de10, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:51.243014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4890bd-6e10-472b-9fdc-641bf0f557f6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:51.263277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1161/2315] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:49:51 2022] PUT /identity/v3/projects/4c25a16ad5454e1b805f08036c76eb4f/users/8ead692343d24d709bde96d0be60de10/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:49:51.269345 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56e214-8ab5-4b99-a8d4-082e730131b3 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:51.286518 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56e214-8ab5-4b99-a8d4-082e730131b3 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:51.292865 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:49:51.378200 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c6cc0-0584-41bd-9b8e-59cb6bfabc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:51.379219 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c6cc0-0584-41bd-9b8e-59cb6bfabc9a admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:51.379695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c6cc0-0584-41bd-9b8e-59cb6bfabc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:51.380144 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c6cc0-0584-41bd-9b8e-59cb6bfabc9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:51.381329 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c6cc0-0584-41bd-9b8e-59cb6bfabc9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:51.384812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c6cc0-0584-41bd-9b8e-59cb6bfabc9a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:51.402450 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1155/2316] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:49:51.408346 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4890bd-6e10-472b-9fdc-641bf0f557f6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:51.427154 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4890bd-6e10-472b-9fdc-641bf0f557f6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:51.435561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815556af-013a-49b0-8d76-092c4ff8a01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:51.436538 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815556af-013a-49b0-8d76-092c4ff8a01c admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:51.437008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815556af-013a-49b0-8d76-092c4ff8a01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:51.437502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815556af-013a-49b0-8d76-092c4ff8a01c admin admin] PATH_INFO: `/v3/projects/4c25a16ad5454e1b805f08036c76eb4f/users/8ead692343d24d709bde96d0be60de10/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:51.441116 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815556af-013a-49b0-8d76-092c4ff8a01c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c25a16ad5454e1b805f08036c76eb4f, user_id=8ead692343d24d709bde96d0be60de10, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:51.446835 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815556af-013a-49b0-8d76-092c4ff8a01c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:51.464960 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1162/2317] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:49:51 2022] PUT /identity/v3/projects/4c25a16ad5454e1b805f08036c76eb4f/users/8ead692343d24d709bde96d0be60de10/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:49:51.475729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efd11b-0af9-4930-97b5-77e32bb635dc None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:51.476018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efd11b-0af9-4930-97b5-77e32bb635dc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:51.476326 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efd11b-0af9-4930-97b5-77e32bb635dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:51.513450 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-44efd11b-0af9-4930-97b5-77e32bb635dc None None] MFA Rules not processed for user `8ead692343d24d709bde96d0be60de10`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:49:51.514748 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-44efd11b-0af9-4930-97b5-77e32bb635dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:51.557554 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815556af-013a-49b0-8d76-092c4ff8a01c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:51.559512 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-815556af-013a-49b0-8d76-092c4ff8a01c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:51.611545 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:49:51.621237 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1156/2318] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:49:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:49:51.634807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44efd11b-0af9-4930-97b5-77e32bb635dc None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:51.636445 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-44efd11b-0af9-4930-97b5-77e32bb635dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:51.672250 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815556af-013a-49b0-8d76-092c4ff8a01c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:51.681523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e3b76-f343-4930-97ef-4ba14198db79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:51.682409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e3b76-f343-4930-97ef-4ba14198db79 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:51.682788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e3b76-f343-4930-97ef-4ba14198db79 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:51.683191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e3b76-f343-4930-97ef-4ba14198db79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:51.685236 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7c6e3b76-f343-4930-97ef-4ba14198db79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:49:51.700713 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e3b76-f343-4930-97ef-4ba14198db79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:51.702691 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e3b76-f343-4930-97ef-4ba14198db79 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:51.711834 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44efd11b-0af9-4930-97b5-77e32bb635dc None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:51.714850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1163/2319] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:49:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:49:51.715715 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620c519-7645-4773-8079-b07bdd292610 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:51.716351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620c519-7645-4773-8079-b07bdd292610 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:51.716660 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620c519-7645-4773-8079-b07bdd292610 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:51.716970 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620c519-7645-4773-8079-b07bdd292610 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:51.728053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620c519-7645-4773-8079-b07bdd292610 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:51.729891 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620c519-7645-4773-8079-b07bdd292610 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:51.742858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1157/2320] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:49:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:49:51.844310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e3b76-f343-4930-97ef-4ba14198db79 service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:49:51.856099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e3b76-f343-4930-97ef-4ba14198db79 service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:49:51.860914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4443f-bc8e-4753-82cb-d70f08560011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:49:51.861736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4443f-bc8e-4753-82cb-d70f08560011 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:49:51.862015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4443f-bc8e-4753-82cb-d70f08560011 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:49:51.862283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4443f-bc8e-4753-82cb-d70f08560011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:49:51.933232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4443f-bc8e-4753-82cb-d70f08560011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:49:51.934611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4443f-bc8e-4753-82cb-d70f08560011 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:49:51.945203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1164/2321] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:49:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:01.367124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620c519-7645-4773-8079-b07bdd292610 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:01.400884 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:01.430195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620c519-7645-4773-8079-b07bdd292610 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:01.435308 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3a037-6408-45af-a816-9e80394363d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:01.435955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3a037-6408-45af-a816-9e80394363d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:01.436236 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3a037-6408-45af-a816-9e80394363d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:01.436508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3a037-6408-45af-a816-9e80394363d9 admin admin] PATH_INFO: `/v3/users/170faab1c46045fba061143dfe1c0872` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:01.437686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3a037-6408-45af-a816-9e80394363d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=170faab1c46045fba061143dfe1c0872)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:01.439593 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3a037-6408-45af-a816-9e80394363d9 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:01.488253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-71a3a037-6408-45af-a816-9e80394363d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '170faab1c46045fba061143dfe1c0872'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:01.495217 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-71a3a037-6408-45af-a816-9e80394363d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '170faab1c46045fba061143dfe1c0872'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:01.502542 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-71a3a037-6408-45af-a816-9e80394363d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '170faab1c46045fba061143dfe1c0872'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:01.503299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-71a3a037-6408-45af-a816-9e80394363d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '170faab1c46045fba061143dfe1c0872'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:01.503948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-71a3a037-6408-45af-a816-9e80394363d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '170faab1c46045fba061143dfe1c0872'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:01.522071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1158/2322] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:01 2022] DELETE /identity/v3/users/170faab1c46045fba061143dfe1c0872 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:50:01.527188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0237c-ed60-46c4-b74b-c974ca4a1fe2 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:01.527688 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0237c-ed60-46c4-b74b-c974ca4a1fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:01.527795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0237c-ed60-46c4-b74b-c974ca4a1fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:01.546357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a4a0237c-ed60-46c4-b74b-c974ca4a1fe2 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:01.547530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a4a0237c-ed60-46c4-b74b-c974ca4a1fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:01.606222 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1165/2323] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:01.616235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3a037-6408-45af-a816-9e80394363d9 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:01.617676 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-71a3a037-6408-45af-a816-9e80394363d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:01.675019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3a037-6408-45af-a816-9e80394363d9 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:01.678645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab420a40-407f-41c9-9f55-6c5127b221bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:01.679272 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab420a40-407f-41c9-9f55-6c5127b221bf service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:01.679557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab420a40-407f-41c9-9f55-6c5127b221bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:01.679833 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab420a40-407f-41c9-9f55-6c5127b221bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:01.689082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab420a40-407f-41c9-9f55-6c5127b221bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:01.690546 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab420a40-407f-41c9-9f55-6c5127b221bf service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:01.700405 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1159/2324] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:01.975189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0237c-ed60-46c4-b74b-c974ca4a1fe2 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:01.976646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a4a0237c-ed60-46c4-b74b-c974ca4a1fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:01.991390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0237c-ed60-46c4-b74b-c974ca4a1fe2 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:01.997264 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:01.998783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:01.999255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:01.999541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2 admin admin] PATH_INFO: `/v3/projects/7e02d017fdd54381bbc2d3793182e667` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:02.000774 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e02d017fdd54381bbc2d3793182e667)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:02.002739 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:02.053156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e02d017fdd54381bbc2d3793182e667', 'request_id': 'req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:02.061917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e02d017fdd54381bbc2d3793182e667', 'request_id': 'req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:02.063277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:02.070184 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e02d017fdd54381bbc2d3793182e667', 'request_id': 'req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:02.083257 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2 admin admin] The token cache is being invalidate because project 7e02d017fdd54381bbc2d3793182e667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:02.083548 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:02.084200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:02.085815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1166/2325] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:01 2022] DELETE /identity/v3/projects/7e02d017fdd54381bbc2d3793182e667 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:02.089796 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab420a40-407f-41c9-9f55-6c5127b221bf service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:02.099317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab420a40-407f-41c9-9f55-6c5127b221bf service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:02.146066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:02.156451 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294aac3b-fae6-4538-b25e-c30cc7b0942c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:02.157265 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294aac3b-fae6-4538-b25e-c30cc7b0942c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:02.157757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294aac3b-fae6-4538-b25e-c30cc7b0942c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:02.158102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294aac3b-fae6-4538-b25e-c30cc7b0942c admin admin] PATH_INFO: `/v3/users/8ead692343d24d709bde96d0be60de10` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:02.159523 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294aac3b-fae6-4538-b25e-c30cc7b0942c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ead692343d24d709bde96d0be60de10)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:02.161588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294aac3b-fae6-4538-b25e-c30cc7b0942c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:02.207425 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-294aac3b-fae6-4538-b25e-c30cc7b0942c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ead692343d24d709bde96d0be60de10'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:02.214307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-294aac3b-fae6-4538-b25e-c30cc7b0942c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ead692343d24d709bde96d0be60de10'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:02.221456 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-294aac3b-fae6-4538-b25e-c30cc7b0942c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ead692343d24d709bde96d0be60de10'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:02.222114 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-294aac3b-fae6-4538-b25e-c30cc7b0942c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ead692343d24d709bde96d0be60de10'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:02.222726 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-294aac3b-fae6-4538-b25e-c30cc7b0942c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ead692343d24d709bde96d0be60de10'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:02.243944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1160/2326] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:02 2022] DELETE /identity/v3/users/8ead692343d24d709bde96d0be60de10 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:02.545977 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:02.547485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:02.562793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ffd5c-bcf2-42d8-9d65-21f4c8ddc0b2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:02.622206 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6bbb5-35d3-49d8-8b0b-237ac1195c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:02.622773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6bbb5-35d3-49d8-8b0b-237ac1195c0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:02.623050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6bbb5-35d3-49d8-8b0b-237ac1195c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:02.623322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6bbb5-35d3-49d8-8b0b-237ac1195c0a admin admin] PATH_INFO: `/v3/projects/4c25a16ad5454e1b805f08036c76eb4f` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:02.624485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6bbb5-35d3-49d8-8b0b-237ac1195c0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c25a16ad5454e1b805f08036c76eb4f)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:02.626457 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6bbb5-35d3-49d8-8b0b-237ac1195c0a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:02.673120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4ea6bbb5-35d3-49d8-8b0b-237ac1195c0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c25a16ad5454e1b805f08036c76eb4f', 'request_id': 'req-4ea6bbb5-35d3-49d8-8b0b-237ac1195c0a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:02.680358 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4ea6bbb5-35d3-49d8-8b0b-237ac1195c0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c25a16ad5454e1b805f08036c76eb4f', 'request_id': 'req-4ea6bbb5-35d3-49d8-8b0b-237ac1195c0a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:02.687523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4ea6bbb5-35d3-49d8-8b0b-237ac1195c0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c25a16ad5454e1b805f08036c76eb4f', 'request_id': 'req-4ea6bbb5-35d3-49d8-8b0b-237ac1195c0a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:02.698022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4ea6bbb5-35d3-49d8-8b0b-237ac1195c0a admin admin] The token cache is being invalidate because project 4c25a16ad5454e1b805f08036c76eb4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:02.698312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4ea6bbb5-35d3-49d8-8b0b-237ac1195c0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:02.698962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4ea6bbb5-35d3-49d8-8b0b-237ac1195c0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:02.700479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1167/2327] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:02 2022] DELETE /identity/v3/projects/4c25a16ad5454e1b805f08036c76eb4f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:02.705300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60c562-5b6e-46cf-ac4f-2173e79b2c6f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:02.705795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60c562-5b6e-46cf-ac4f-2173e79b2c6f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:02.705957 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60c562-5b6e-46cf-ac4f-2173e79b2c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:02.725701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-dd60c562-5b6e-46cf-ac4f-2173e79b2c6f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:02.726911 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-dd60c562-5b6e-46cf-ac4f-2173e79b2c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:02.786514 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1161/2328] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:02.795107 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cab853-6822-4a29-a5d7-f2a064f43f40 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:02.795395 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cab853-6822-4a29-a5d7-f2a064f43f40 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:02.795675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cab853-6822-4a29-a5d7-f2a064f43f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:02.815313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f1cab853-6822-4a29-a5d7-f2a064f43f40 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:02.816538 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f1cab853-6822-4a29-a5d7-f2a064f43f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:02.825067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1168/2329] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:02.829514 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60c562-5b6e-46cf-ac4f-2173e79b2c6f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:02.839861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60c562-5b6e-46cf-ac4f-2173e79b2c6f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:02.844547 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a20e71-c400-4c5d-850f-cda475096c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:02.845180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a20e71-c400-4c5d-850f-cda475096c5d admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:02.845487 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a20e71-c400-4c5d-850f-cda475096c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:02.845774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a20e71-c400-4c5d-850f-cda475096c5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:02.846496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a20e71-c400-4c5d-850f-cda475096c5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:50:02.846806 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a20e71-c400-4c5d-850f-cda475096c5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:02.848430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a20e71-c400-4c5d-850f-cda475096c5d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:02.863842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1162/2330] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:50:02.868796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cab853-6822-4a29-a5d7-f2a064f43f40 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:02.878198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cab853-6822-4a29-a5d7-f2a064f43f40 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:02.882798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49a5f1-9d30-4e45-9c84-0e9c26156253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:02.883404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49a5f1-9d30-4e45-9c84-0e9c26156253 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:02.884174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49a5f1-9d30-4e45-9c84-0e9c26156253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:02.884450 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49a5f1-9d30-4e45-9c84-0e9c26156253 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:02.885104 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49a5f1-9d30-4e45-9c84-0e9c26156253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:02.887017 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49a5f1-9d30-4e45-9c84-0e9c26156253 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:02.915419 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1169/2331] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:02 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:02.921397 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a20e71-c400-4c5d-850f-cda475096c5d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:02.931363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a20e71-c400-4c5d-850f-cda475096c5d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:02.988363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cd8e6-de5b-4563-9a86-8ed3026bad60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:02.989254 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cd8e6-de5b-4563-9a86-8ed3026bad60 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:02.989564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cd8e6-de5b-4563-9a86-8ed3026bad60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:02.989844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cd8e6-de5b-4563-9a86-8ed3026bad60 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:02.990508 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789cd8e6-de5b-4563-9a86-8ed3026bad60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:02.992429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789cd8e6-de5b-4563-9a86-8ed3026bad60 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:03.000481 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:03.016614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1163/2332] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:02 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:03.020882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49a5f1-9d30-4e45-9c84-0e9c26156253 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:03.032091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49a5f1-9d30-4e45-9c84-0e9c26156253 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:03.036860 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e5f66-12a3-4504-b4c0-7fc84824c235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:03.037553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e5f66-12a3-4504-b4c0-7fc84824c235 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:03.037776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e5f66-12a3-4504-b4c0-7fc84824c235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:03.038046 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e5f66-12a3-4504-b4c0-7fc84824c235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:03.038762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e5f66-12a3-4504-b4c0-7fc84824c235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:03.040344 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e5f66-12a3-4504-b4c0-7fc84824c235 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:03.050472 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1170/2333] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:03.054621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cd8e6-de5b-4563-9a86-8ed3026bad60 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:03.064511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cd8e6-de5b-4563-9a86-8ed3026bad60 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:03.069030 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab680b2e-fe39-4443-91dd-24e0463c27a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:03.069731 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab680b2e-fe39-4443-91dd-24e0463c27a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:03.070014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab680b2e-fe39-4443-91dd-24e0463c27a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:03.070288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab680b2e-fe39-4443-91dd-24e0463c27a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:03.070970 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab680b2e-fe39-4443-91dd-24e0463c27a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:03.073067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab680b2e-fe39-4443-91dd-24e0463c27a2 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:03.083526 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1164/2334] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:03.088421 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e5f66-12a3-4504-b4c0-7fc84824c235 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:03.099250 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e5f66-12a3-4504-b4c0-7fc84824c235 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:03.104333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725c388-ea72-443f-baf9-7acceb48c2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:03.105097 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725c388-ea72-443f-baf9-7acceb48c2ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:03.105448 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725c388-ea72-443f-baf9-7acceb48c2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:03.105826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725c388-ea72-443f-baf9-7acceb48c2ed admin admin] PATH_INFO: `/v3/projects/35e31f345b3a43928636c16c722d9b29/users/9c9597950d6a4d1082f074a5962187bd/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:03.123519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725c388-ea72-443f-baf9-7acceb48c2ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e31f345b3a43928636c16c722d9b29, user_id=9c9597950d6a4d1082f074a5962187bd, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:03.126191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725c388-ea72-443f-baf9-7acceb48c2ed admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:03.140542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1171/2335] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:03 2022] PUT /identity/v3/projects/35e31f345b3a43928636c16c722d9b29/users/9c9597950d6a4d1082f074a5962187bd/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:03.158244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d95e2f-7650-46d6-a3a8-12de63d4b5ee None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:03.158529 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d95e2f-7650-46d6-a3a8-12de63d4b5ee None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:03.158854 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d95e2f-7650-46d6-a3a8-12de63d4b5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:03.186429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d0d95e2f-7650-46d6-a3a8-12de63d4b5ee None None] MFA Rules not processed for user `9c9597950d6a4d1082f074a5962187bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:03.187651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d0d95e2f-7650-46d6-a3a8-12de63d4b5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:03.259457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1165/2336] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:03.263838 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725c388-ea72-443f-baf9-7acceb48c2ed admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:03.276582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:03.327486 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725c388-ea72-443f-baf9-7acceb48c2ed admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:03.332165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1645d-5a08-499f-a51c-4eb3b7572bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:03.332788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1645d-5a08-499f-a51c-4eb3b7572bca admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:03.333070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1645d-5a08-499f-a51c-4eb3b7572bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:03.333349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1645d-5a08-499f-a51c-4eb3b7572bca admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:03.334067 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1645d-5a08-499f-a51c-4eb3b7572bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:03.335966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1645d-5a08-499f-a51c-4eb3b7572bca admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:03.362077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1172/2337] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:03 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:03.366442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d95e2f-7650-46d6-a3a8-12de63d4b5ee None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:03.377513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d95e2f-7650-46d6-a3a8-12de63d4b5ee None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:03.433380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e6262-a8f5-40dd-8113-b7982046bd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:03.433952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e6262-a8f5-40dd-8113-b7982046bd8d admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:03.434237 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e6262-a8f5-40dd-8113-b7982046bd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:03.434513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e6262-a8f5-40dd-8113-b7982046bd8d admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:03.435187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e6262-a8f5-40dd-8113-b7982046bd8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:03.437099 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e6262-a8f5-40dd-8113-b7982046bd8d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:03.457702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1166/2338] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:03 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:03.461731 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1645d-5a08-499f-a51c-4eb3b7572bca admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:03.472696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1645d-5a08-499f-a51c-4eb3b7572bca admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:03.477602 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecd7b3-3211-4e73-894a-65860df0f419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:03.478234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecd7b3-3211-4e73-894a-65860df0f419 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:03.478520 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecd7b3-3211-4e73-894a-65860df0f419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:03.478798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecd7b3-3211-4e73-894a-65860df0f419 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:03.479508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecd7b3-3211-4e73-894a-65860df0f419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:03.481164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecd7b3-3211-4e73-894a-65860df0f419 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:03.492492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1173/2339] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:03.497229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e6262-a8f5-40dd-8113-b7982046bd8d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:03.507494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e6262-a8f5-40dd-8113-b7982046bd8d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:03.512949 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46f9e0-4b4a-4b21-900d-b3697ffaf521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:03.513677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46f9e0-4b4a-4b21-900d-b3697ffaf521 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:03.513963 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46f9e0-4b4a-4b21-900d-b3697ffaf521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:03.514241 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46f9e0-4b4a-4b21-900d-b3697ffaf521 admin admin] PATH_INFO: `/v3/projects/e5bbdbf3248044eea716316d1636f2d2/users/824560332ca9441cbaa2791de7ecf094/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:03.529617 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46f9e0-4b4a-4b21-900d-b3697ffaf521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5bbdbf3248044eea716316d1636f2d2, user_id=824560332ca9441cbaa2791de7ecf094, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:03.532322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46f9e0-4b4a-4b21-900d-b3697ffaf521 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:03.546205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1167/2340] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:03 2022] PUT /identity/v3/projects/e5bbdbf3248044eea716316d1636f2d2/users/824560332ca9441cbaa2791de7ecf094/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:03.550402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecd7b3-3211-4e73-894a-65860df0f419 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:03.570287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecd7b3-3211-4e73-894a-65860df0f419 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:03.641131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea54c56-18e7-479f-bf88-ca71cb7c5926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:03.641970 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea54c56-18e7-479f-bf88-ca71cb7c5926 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:03.642261 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea54c56-18e7-479f-bf88-ca71cb7c5926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:03.642557 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea54c56-18e7-479f-bf88-ca71cb7c5926 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:03.643289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea54c56-18e7-479f-bf88-ca71cb7c5926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:03.644991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea54c56-18e7-479f-bf88-ca71cb7c5926 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:03.656549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1174/2341] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:50:03.660781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46f9e0-4b4a-4b21-900d-b3697ffaf521 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:03.671977 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46f9e0-4b4a-4b21-900d-b3697ffaf521 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:03.676682 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073cae0-54b7-4a27-9cf0-f9ee23c7484c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:03.677351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073cae0-54b7-4a27-9cf0-f9ee23c7484c admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:03.677670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073cae0-54b7-4a27-9cf0-f9ee23c7484c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:03.677951 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073cae0-54b7-4a27-9cf0-f9ee23c7484c admin admin] PATH_INFO: `/v3/projects/e5bbdbf3248044eea716316d1636f2d2/users/824560332ca9441cbaa2791de7ecf094/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:03.679897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073cae0-54b7-4a27-9cf0-f9ee23c7484c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5bbdbf3248044eea716316d1636f2d2, user_id=824560332ca9441cbaa2791de7ecf094, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:03.683230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073cae0-54b7-4a27-9cf0-f9ee23c7484c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:03.694565 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1168/2342] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:03 2022] PUT /identity/v3/projects/e5bbdbf3248044eea716316d1636f2d2/users/824560332ca9441cbaa2791de7ecf094/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:03.705069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d2158-34d0-4c60-bcde-97d11d89108d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:03.705364 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d2158-34d0-4c60-bcde-97d11d89108d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:03.706517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d2158-34d0-4c60-bcde-97d11d89108d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:03.734272 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-596d2158-34d0-4c60-bcde-97d11d89108d None None] MFA Rules not processed for user `824560332ca9441cbaa2791de7ecf094`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:03.735488 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-596d2158-34d0-4c60-bcde-97d11d89108d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:03.805190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1175/2343] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:03.816047 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073cae0-54b7-4a27-9cf0-f9ee23c7484c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:03.817760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7073cae0-54b7-4a27-9cf0-f9ee23c7484c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:03.842037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:03.877173 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073cae0-54b7-4a27-9cf0-f9ee23c7484c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:03.880837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acafcaa-500d-4d83-b13d-753518ff8dac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:03.881452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acafcaa-500d-4d83-b13d-753518ff8dac service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:03.881763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acafcaa-500d-4d83-b13d-753518ff8dac service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:03.882038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acafcaa-500d-4d83-b13d-753518ff8dac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:03.891887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acafcaa-500d-4d83-b13d-753518ff8dac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:03.893320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acafcaa-500d-4d83-b13d-753518ff8dac service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:03.903451 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1169/2344] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:05.291534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d2158-34d0-4c60-bcde-97d11d89108d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:05.299920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:05.355348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d2158-34d0-4c60-bcde-97d11d89108d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:05.359880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010274fd-4b6e-426b-81b7-c1582deb94ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:05.360563 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010274fd-4b6e-426b-81b7-c1582deb94ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:05.360836 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010274fd-4b6e-426b-81b7-c1582deb94ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:05.361203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010274fd-4b6e-426b-81b7-c1582deb94ca admin admin] PATH_INFO: `/v3/users/9c9597950d6a4d1082f074a5962187bd` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:05.362455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010274fd-4b6e-426b-81b7-c1582deb94ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c9597950d6a4d1082f074a5962187bd)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:05.364631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010274fd-4b6e-426b-81b7-c1582deb94ca admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:05.412903 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-010274fd-4b6e-426b-81b7-c1582deb94ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c9597950d6a4d1082f074a5962187bd'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:05.424912 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-010274fd-4b6e-426b-81b7-c1582deb94ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c9597950d6a4d1082f074a5962187bd'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:05.425601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-010274fd-4b6e-426b-81b7-c1582deb94ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c9597950d6a4d1082f074a5962187bd'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:05.433590 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-010274fd-4b6e-426b-81b7-c1582deb94ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c9597950d6a4d1082f074a5962187bd'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:05.440447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-010274fd-4b6e-426b-81b7-c1582deb94ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c9597950d6a4d1082f074a5962187bd'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:05.446389 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1176/2345] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:05 2022] DELETE /identity/v3/users/9c9597950d6a4d1082f074a5962187bd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:05.450992 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce5915-ba88-4306-a0db-9b8fef45d1bc None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:05.451510 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce5915-ba88-4306-a0db-9b8fef45d1bc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:05.451598 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce5915-ba88-4306-a0db-9b8fef45d1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:05.471329 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8cce5915-ba88-4306-a0db-9b8fef45d1bc None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:05.472548 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8cce5915-ba88-4306-a0db-9b8fef45d1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:05.537748 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1170/2346] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:05.549810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010274fd-4b6e-426b-81b7-c1582deb94ca admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:05.551271 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-010274fd-4b6e-426b-81b7-c1582deb94ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:05.614248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010274fd-4b6e-426b-81b7-c1582deb94ca admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:05.617916 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65dc94-7f2e-42c6-b4be-653c90ed8c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:05.618537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65dc94-7f2e-42c6-b4be-653c90ed8c17 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:05.618817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65dc94-7f2e-42c6-b4be-653c90ed8c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:05.619085 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65dc94-7f2e-42c6-b4be-653c90ed8c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:05.628758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65dc94-7f2e-42c6-b4be-653c90ed8c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:05.630206 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65dc94-7f2e-42c6-b4be-653c90ed8c17 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:05.640476 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1177/2347] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:05.926774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce5915-ba88-4306-a0db-9b8fef45d1bc None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:05.928500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8cce5915-ba88-4306-a0db-9b8fef45d1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:05.947372 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce5915-ba88-4306-a0db-9b8fef45d1bc None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:05.962007 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70c00a-fdba-431b-ac40-8a2be2635f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:05.962763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70c00a-fdba-431b-ac40-8a2be2635f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:05.963044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70c00a-fdba-431b-ac40-8a2be2635f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:05.963321 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70c00a-fdba-431b-ac40-8a2be2635f6b admin admin] PATH_INFO: `/v3/projects/35e31f345b3a43928636c16c722d9b29` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:05.965722 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70c00a-fdba-431b-ac40-8a2be2635f6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=35e31f345b3a43928636c16c722d9b29)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:05.969669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70c00a-fdba-431b-ac40-8a2be2635f6b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:06.038004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4a70c00a-fdba-431b-ac40-8a2be2635f6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35e31f345b3a43928636c16c722d9b29', 'request_id': 'req-4a70c00a-fdba-431b-ac40-8a2be2635f6b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:06.044093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4a70c00a-fdba-431b-ac40-8a2be2635f6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35e31f345b3a43928636c16c722d9b29', 'request_id': 'req-4a70c00a-fdba-431b-ac40-8a2be2635f6b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:06.052004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4a70c00a-fdba-431b-ac40-8a2be2635f6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35e31f345b3a43928636c16c722d9b29', 'request_id': 'req-4a70c00a-fdba-431b-ac40-8a2be2635f6b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:06.062861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4a70c00a-fdba-431b-ac40-8a2be2635f6b admin admin] The token cache is being invalidate because project 35e31f345b3a43928636c16c722d9b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:06.063143 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4a70c00a-fdba-431b-ac40-8a2be2635f6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:06.063777 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4a70c00a-fdba-431b-ac40-8a2be2635f6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:06.065286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1171/2348] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:05 2022] DELETE /identity/v3/projects/35e31f345b3a43928636c16c722d9b29 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:06.070333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65dc94-7f2e-42c6-b4be-653c90ed8c17 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:06.084503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65dc94-7f2e-42c6-b4be-653c90ed8c17 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:06.112999 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:06.150699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2328b-6da5-4670-911a-00bca7e6a09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:06.151719 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2328b-6da5-4670-911a-00bca7e6a09e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:06.152016 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2328b-6da5-4670-911a-00bca7e6a09e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:06.152294 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2328b-6da5-4670-911a-00bca7e6a09e admin admin] PATH_INFO: `/v3/users/824560332ca9441cbaa2791de7ecf094` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:06.153715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2328b-6da5-4670-911a-00bca7e6a09e admin admin] RBAC: Authorizing `identity:delete_user(user_id=824560332ca9441cbaa2791de7ecf094)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:06.155712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2328b-6da5-4670-911a-00bca7e6a09e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:06.214327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-26e2328b-6da5-4670-911a-00bca7e6a09e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '824560332ca9441cbaa2791de7ecf094'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:06.226656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-26e2328b-6da5-4670-911a-00bca7e6a09e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '824560332ca9441cbaa2791de7ecf094'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:06.227997 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-26e2328b-6da5-4670-911a-00bca7e6a09e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '824560332ca9441cbaa2791de7ecf094'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:06.236819 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-26e2328b-6da5-4670-911a-00bca7e6a09e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '824560332ca9441cbaa2791de7ecf094'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:06.245661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-26e2328b-6da5-4670-911a-00bca7e6a09e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '824560332ca9441cbaa2791de7ecf094'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:06.252200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1178/2349] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:06 2022] DELETE /identity/v3/users/824560332ca9441cbaa2791de7ecf094 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:06.542560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70c00a-fdba-431b-ac40-8a2be2635f6b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:06.544220 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4a70c00a-fdba-431b-ac40-8a2be2635f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:06.558515 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70c00a-fdba-431b-ac40-8a2be2635f6b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:06.614873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:06.629697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800d483-7469-47f3-8738-8ed6e91026f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:06.630356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800d483-7469-47f3-8738-8ed6e91026f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:06.630638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800d483-7469-47f3-8738-8ed6e91026f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:06.630936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800d483-7469-47f3-8738-8ed6e91026f0 admin admin] PATH_INFO: `/v3/projects/e5bbdbf3248044eea716316d1636f2d2` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:06.632094 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5800d483-7469-47f3-8738-8ed6e91026f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5bbdbf3248044eea716316d1636f2d2)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:06.634597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5800d483-7469-47f3-8738-8ed6e91026f0 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:06.684103 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5800d483-7469-47f3-8738-8ed6e91026f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5bbdbf3248044eea716316d1636f2d2', 'request_id': 'req-5800d483-7469-47f3-8738-8ed6e91026f0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:06.691059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5800d483-7469-47f3-8738-8ed6e91026f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5bbdbf3248044eea716316d1636f2d2', 'request_id': 'req-5800d483-7469-47f3-8738-8ed6e91026f0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:06.700457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5800d483-7469-47f3-8738-8ed6e91026f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5bbdbf3248044eea716316d1636f2d2', 'request_id': 'req-5800d483-7469-47f3-8738-8ed6e91026f0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:06.714266 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5800d483-7469-47f3-8738-8ed6e91026f0 admin admin] The token cache is being invalidate because project e5bbdbf3248044eea716316d1636f2d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:06.714558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5800d483-7469-47f3-8738-8ed6e91026f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:06.715208 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5800d483-7469-47f3-8738-8ed6e91026f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:06.716752 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1172/2350] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:06 2022] DELETE /identity/v3/projects/e5bbdbf3248044eea716316d1636f2d2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:06.721759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4664b-3869-4eaa-8221-2881997c8c23 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:06.722301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4664b-3869-4eaa-8221-2881997c8c23 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:06.722411 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4664b-3869-4eaa-8221-2881997c8c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:06.742688 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-57d4664b-3869-4eaa-8221-2881997c8c23 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:06.744071 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-57d4664b-3869-4eaa-8221-2881997c8c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:06.806764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1179/2351] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:06.815962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1de589-44d9-46b2-b8fb-a6508182fb7a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:06.816249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1de589-44d9-46b2-b8fb-a6508182fb7a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:06.816531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1de589-44d9-46b2-b8fb-a6508182fb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:06.838327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7f1de589-44d9-46b2-b8fb-a6508182fb7a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:06.839589 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7f1de589-44d9-46b2-b8fb-a6508182fb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:06.847820 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1173/2352] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:06.852187 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4664b-3869-4eaa-8221-2881997c8c23 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:06.862069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4664b-3869-4eaa-8221-2881997c8c23 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:06.866959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f756e-4a85-42ab-b4b3-87ee8ab53ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:06.867504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f756e-4a85-42ab-b4b3-87ee8ab53ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:06.867881 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f756e-4a85-42ab-b4b3-87ee8ab53ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:06.868060 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f756e-4a85-42ab-b4b3-87ee8ab53ae3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:06.868735 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f756e-4a85-42ab-b4b3-87ee8ab53ae3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:50:06.869035 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f756e-4a85-42ab-b4b3-87ee8ab53ae3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:06.870761 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f756e-4a85-42ab-b4b3-87ee8ab53ae3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:06.884722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1180/2353] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:50:06.889133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1de589-44d9-46b2-b8fb-a6508182fb7a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:06.901408 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1de589-44d9-46b2-b8fb-a6508182fb7a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:06.906135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc6865-af43-49c6-89c3-5292c204862c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:06.906751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc6865-af43-49c6-89c3-5292c204862c admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:06.907031 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc6865-af43-49c6-89c3-5292c204862c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:06.907304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc6865-af43-49c6-89c3-5292c204862c admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:06.907965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc6865-af43-49c6-89c3-5292c204862c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:06.909896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc6865-af43-49c6-89c3-5292c204862c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:06.938805 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1174/2354] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:06 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:06.941924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f756e-4a85-42ab-b4b3-87ee8ab53ae3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:06.956432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f756e-4a85-42ab-b4b3-87ee8ab53ae3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:06.991411 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:07.019136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b6448-3da3-48a5-921b-d62c339109a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:07.019767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b6448-3da3-48a5-921b-d62c339109a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:07.020163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b6448-3da3-48a5-921b-d62c339109a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:07.020675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b6448-3da3-48a5-921b-d62c339109a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:07.021741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b6448-3da3-48a5-921b-d62c339109a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:07.023975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b6448-3da3-48a5-921b-d62c339109a0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:07.052612 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1181/2355] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:06 2022] POST /identity/v3/users => generated 296 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:07.057291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc6865-af43-49c6-89c3-5292c204862c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:07.067777 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc6865-af43-49c6-89c3-5292c204862c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:07.072914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed5867-3eec-436f-94fe-3ab52b2542a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:07.073563 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed5867-3eec-436f-94fe-3ab52b2542a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:07.073853 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed5867-3eec-436f-94fe-3ab52b2542a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:07.074119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed5867-3eec-436f-94fe-3ab52b2542a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:07.074811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed5867-3eec-436f-94fe-3ab52b2542a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:07.076398 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed5867-3eec-436f-94fe-3ab52b2542a5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:07.086604 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1175/2356] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:07.090726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b6448-3da3-48a5-921b-d62c339109a0 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:07.100851 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b6448-3da3-48a5-921b-d62c339109a0 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:07.106010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25922ee-e6f2-4d37-965d-8aeb652f2fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:07.106629 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25922ee-e6f2-4d37-965d-8aeb652f2fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:07.106910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25922ee-e6f2-4d37-965d-8aeb652f2fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:07.107187 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25922ee-e6f2-4d37-965d-8aeb652f2fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:07.107890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25922ee-e6f2-4d37-965d-8aeb652f2fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:07.109495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25922ee-e6f2-4d37-965d-8aeb652f2fb2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:07.120293 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1182/2357] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:07.124406 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed5867-3eec-436f-94fe-3ab52b2542a5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:07.134999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed5867-3eec-436f-94fe-3ab52b2542a5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:07.139634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b070ce-09c2-4da8-b071-c72baaaf4c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:07.140292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b070ce-09c2-4da8-b071-c72baaaf4c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:07.140570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b070ce-09c2-4da8-b071-c72baaaf4c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:07.140843 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b070ce-09c2-4da8-b071-c72baaaf4c0b admin admin] PATH_INFO: `/v3/projects/ebf3590f807c4d299f5f8e37485dc6fe/users/a776c6564bf14b708c6ef3f1133d76db/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:07.155448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b070ce-09c2-4da8-b071-c72baaaf4c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebf3590f807c4d299f5f8e37485dc6fe, user_id=a776c6564bf14b708c6ef3f1133d76db, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:07.158139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b070ce-09c2-4da8-b071-c72baaaf4c0b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:07.171588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1176/2358] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:07 2022] PUT /identity/v3/projects/ebf3590f807c4d299f5f8e37485dc6fe/users/a776c6564bf14b708c6ef3f1133d76db/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:50:07.182540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ade9c-ca7b-4d55-95c0-6467a43400a7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:07.182850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ade9c-ca7b-4d55-95c0-6467a43400a7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:07.183133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ade9c-ca7b-4d55-95c0-6467a43400a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:07.210921 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9c9ade9c-ca7b-4d55-95c0-6467a43400a7 None None] MFA Rules not processed for user `a776c6564bf14b708c6ef3f1133d76db`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:07.212152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9c9ade9c-ca7b-4d55-95c0-6467a43400a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:07.286683 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1183/2359] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:07.291195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b070ce-09c2-4da8-b071-c72baaaf4c0b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:07.365134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b070ce-09c2-4da8-b071-c72baaaf4c0b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:07.370661 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df3987-08e2-4039-bf58-ea2da469777d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:07.371241 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df3987-08e2-4039-bf58-ea2da469777d admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:07.371522 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df3987-08e2-4039-bf58-ea2da469777d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:07.371802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df3987-08e2-4039-bf58-ea2da469777d admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:07.372655 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df3987-08e2-4039-bf58-ea2da469777d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:07.374445 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df3987-08e2-4039-bf58-ea2da469777d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:07.396791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:07.407927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1177/2360] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:07 2022] POST /identity/v3/projects => generated 391 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:07.422429 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ade9c-ca7b-4d55-95c0-6467a43400a7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:07.439571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ade9c-ca7b-4d55-95c0-6467a43400a7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:07.530489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a19cdc-28a9-4a6f-916b-be80d0f127ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:07.531495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a19cdc-28a9-4a6f-916b-be80d0f127ae admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:07.531791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a19cdc-28a9-4a6f-916b-be80d0f127ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:07.533003 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a19cdc-28a9-4a6f-916b-be80d0f127ae admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:07.533003 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a19cdc-28a9-4a6f-916b-be80d0f127ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:07.535267 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a19cdc-28a9-4a6f-916b-be80d0f127ae admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:07.567672 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1184/2361] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:07 2022] POST /identity/v3/users => generated 302 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:07.568810 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df3987-08e2-4039-bf58-ea2da469777d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:07.584525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df3987-08e2-4039-bf58-ea2da469777d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:07.594050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd9a8f-5a33-4db0-8fc8-151712699f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:07.594860 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd9a8f-5a33-4db0-8fc8-151712699f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:07.595224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd9a8f-5a33-4db0-8fc8-151712699f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:07.595782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd9a8f-5a33-4db0-8fc8-151712699f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:07.596580 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd9a8f-5a33-4db0-8fc8-151712699f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:07.598899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd9a8f-5a33-4db0-8fc8-151712699f54 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:07.611909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1178/2362] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:50:07.616302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a19cdc-28a9-4a6f-916b-be80d0f127ae admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:07.632327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a19cdc-28a9-4a6f-916b-be80d0f127ae admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:07.642072 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c84980-86f6-48d1-b0f3-587f810cac02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:07.646642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c84980-86f6-48d1-b0f3-587f810cac02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:07.647105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c84980-86f6-48d1-b0f3-587f810cac02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:07.647528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c84980-86f6-48d1-b0f3-587f810cac02 admin admin] PATH_INFO: `/v3/projects/c9ba3ac78dc543e59dea15505dfc83a5/users/4bc6973b691846f9aad1bcb60676bf1f/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:07.669689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bb9dd-1a84-40cf-9349-f2f88a1d4ff4 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:07.670023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bb9dd-1a84-40cf-9349-f2f88a1d4ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:07.670307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bb9dd-1a84-40cf-9349-f2f88a1d4ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:07.679038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c84980-86f6-48d1-b0f3-587f810cac02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ba3ac78dc543e59dea15505dfc83a5, user_id=4bc6973b691846f9aad1bcb60676bf1f, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:07.687273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c84980-86f6-48d1-b0f3-587f810cac02 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:07.698442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-334bb9dd-1a84-40cf-9349-f2f88a1d4ff4 None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:07.699770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-334bb9dd-1a84-40cf-9349-f2f88a1d4ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:07.707704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1185/2363] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:07 2022] PUT /identity/v3/projects/c9ba3ac78dc543e59dea15505dfc83a5/users/4bc6973b691846f9aad1bcb60676bf1f/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:07.712285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c84980-86f6-48d1-b0f3-587f810cac02 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:07.723394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:07.776965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1179/2364] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 3 10:50:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:07.784989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c84980-86f6-48d1-b0f3-587f810cac02 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:07.794175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba9f38-b851-4966-bd4b-7c634fb8cc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:07.794766 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba9f38-b851-4966-bd4b-7c634fb8cc96 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:07.795303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba9f38-b851-4966-bd4b-7c634fb8cc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:07.795966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba9f38-b851-4966-bd4b-7c634fb8cc96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:07.796388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba9f38-b851-4966-bd4b-7c634fb8cc96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:07.798439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba9f38-b851-4966-bd4b-7c634fb8cc96 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:07.813631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1186/2365] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:07.816872 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bb9dd-1a84-40cf-9349-f2f88a1d4ff4 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:07.831577 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bb9dd-1a84-40cf-9349-f2f88a1d4ff4 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:07.837129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f2ff3-dd36-4a50-8e82-544953494499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:07.838167 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f2ff3-dd36-4a50-8e82-544953494499 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:07.838553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f2ff3-dd36-4a50-8e82-544953494499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:07.838959 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f2ff3-dd36-4a50-8e82-544953494499 admin admin] PATH_INFO: `/v3/projects/c9ba3ac78dc543e59dea15505dfc83a5/users/4bc6973b691846f9aad1bcb60676bf1f/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:07.842776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f2ff3-dd36-4a50-8e82-544953494499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ba3ac78dc543e59dea15505dfc83a5, user_id=4bc6973b691846f9aad1bcb60676bf1f, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:07.845803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f2ff3-dd36-4a50-8e82-544953494499 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:07.854904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba9f38-b851-4966-bd4b-7c634fb8cc96 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:07.860404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1180/2366] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:07 2022] PUT /identity/v3/projects/c9ba3ac78dc543e59dea15505dfc83a5/users/4bc6973b691846f9aad1bcb60676bf1f/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:07.863027 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-71ba9f38-b851-4966-bd4b-7c634fb8cc96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:07.872993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034f022-f32a-40da-bde9-427573d4466c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:07.873322 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034f022-f32a-40da-bde9-427573d4466c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:07.873679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034f022-f32a-40da-bde9-427573d4466c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:07.911651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-9034f022-f32a-40da-bde9-427573d4466c None None] MFA Rules not processed for user `4bc6973b691846f9aad1bcb60676bf1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:07.913113 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9034f022-f32a-40da-bde9-427573d4466c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:08.006438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba9f38-b851-4966-bd4b-7c634fb8cc96 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:08.009617 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1181/2367] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:07 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:08.020928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034f022-f32a-40da-bde9-427573d4466c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:08.022438 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9034f022-f32a-40da-bde9-427573d4466c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:08.083030 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb3a8c-be2a-4cae-95b0-a896f0126991 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:08.083892 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb3a8c-be2a-4cae-95b0-a896f0126991 service swift] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:08.084016 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb3a8c-be2a-4cae-95b0-a896f0126991 service swift] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:08.084282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb3a8c-be2a-4cae-95b0-a896f0126991 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:08.091882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034f022-f32a-40da-bde9-427573d4466c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:08.097353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baafc8d-3151-4e65-b555-f50b7c84b867 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:08.098147 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baafc8d-3151-4e65-b555-f50b7c84b867 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:08.098557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baafc8d-3151-4e65-b555-f50b7c84b867 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:08.099096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baafc8d-3151-4e65-b555-f50b7c84b867 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:08.109552 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baafc8d-3151-4e65-b555-f50b7c84b867 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:08.110976 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baafc8d-3151-4e65-b555-f50b7c84b867 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:08.121417 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1182/2368] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:08 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:50:08.139078 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:08.148859 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb3a8c-be2a-4cae-95b0-a896f0126991 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:08.150528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb3a8c-be2a-4cae-95b0-a896f0126991 service swift] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:08.158890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1187/2369] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:50:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 10:50:08.171650 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baafc8d-3151-4e65-b555-f50b7c84b867 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:08.173105 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0baafc8d-3151-4e65-b555-f50b7c84b867 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:08.187866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baafc8d-3151-4e65-b555-f50b7c84b867 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:08.192169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d6b72-fa0b-4ec8-b1e6-0fcd1695cc6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:08.192918 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d6b72-fa0b-4ec8-b1e6-0fcd1695cc6a service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:08.193343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d6b72-fa0b-4ec8-b1e6-0fcd1695cc6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:08.193706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d6b72-fa0b-4ec8-b1e6-0fcd1695cc6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:08.198775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:08.267331 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d6b72-fa0b-4ec8-b1e6-0fcd1695cc6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:08.268900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d6b72-fa0b-4ec8-b1e6-0fcd1695cc6a service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:08.282060 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1183/2370] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:08 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:12.227242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb3a8c-be2a-4cae-95b0-a896f0126991 service swift] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:12.318799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb3a8c-be2a-4cae-95b0-a896f0126991 service swift] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:12.324069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27267dd4-41b8-40e2-b5dd-39b215e21aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:12.324946 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27267dd4-41b8-40e2-b5dd-39b215e21aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:12.325384 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27267dd4-41b8-40e2-b5dd-39b215e21aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:12.325935 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27267dd4-41b8-40e2-b5dd-39b215e21aeb admin admin] PATH_INFO: `/v3/users/a776c6564bf14b708c6ef3f1133d76db` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:12.327410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27267dd4-41b8-40e2-b5dd-39b215e21aeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a776c6564bf14b708c6ef3f1133d76db)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:12.329549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27267dd4-41b8-40e2-b5dd-39b215e21aeb admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:12.421762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-27267dd4-41b8-40e2-b5dd-39b215e21aeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a776c6564bf14b708c6ef3f1133d76db'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:12.438542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-27267dd4-41b8-40e2-b5dd-39b215e21aeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a776c6564bf14b708c6ef3f1133d76db'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:12.439991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-27267dd4-41b8-40e2-b5dd-39b215e21aeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a776c6564bf14b708c6ef3f1133d76db'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:12.450021 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-27267dd4-41b8-40e2-b5dd-39b215e21aeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a776c6564bf14b708c6ef3f1133d76db'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:12.467632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-27267dd4-41b8-40e2-b5dd-39b215e21aeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a776c6564bf14b708c6ef3f1133d76db'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:12.475497 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1188/2371] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:12 2022] DELETE /identity/v3/users/a776c6564bf14b708c6ef3f1133d76db => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:12.480282 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92850f3-df5b-479d-901e-a001e3923b07 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:12.480821 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92850f3-df5b-479d-901e-a001e3923b07 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:12.480982 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92850f3-df5b-479d-901e-a001e3923b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:12.507343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e92850f3-df5b-479d-901e-a001e3923b07 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:12.508590 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e92850f3-df5b-479d-901e-a001e3923b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:12.593499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1184/2372] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:12.604837 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27267dd4-41b8-40e2-b5dd-39b215e21aeb admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:12.606807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-27267dd4-41b8-40e2-b5dd-39b215e21aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:12.673407 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27267dd4-41b8-40e2-b5dd-39b215e21aeb admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:12.681638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c7fa6-2fc0-422b-aa3a-b883bc079bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:12.682402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c7fa6-2fc0-422b-aa3a-b883bc079bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:12.682699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c7fa6-2fc0-422b-aa3a-b883bc079bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:12.682985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c7fa6-2fc0-422b-aa3a-b883bc079bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:12.698140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c7fa6-2fc0-422b-aa3a-b883bc079bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:12.699820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c7fa6-2fc0-422b-aa3a-b883bc079bd3 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:12.711017 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1189/2373] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:13.079572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92850f3-df5b-479d-901e-a001e3923b07 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:13.081904 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e92850f3-df5b-479d-901e-a001e3923b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:13.110672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92850f3-df5b-479d-901e-a001e3923b07 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:13.126171 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84fb20-111a-475d-b00f-4ddc3eda9420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:13.127045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84fb20-111a-475d-b00f-4ddc3eda9420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:13.127401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84fb20-111a-475d-b00f-4ddc3eda9420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:13.127751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84fb20-111a-475d-b00f-4ddc3eda9420 admin admin] PATH_INFO: `/v3/projects/ebf3590f807c4d299f5f8e37485dc6fe` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:13.132156 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84fb20-111a-475d-b00f-4ddc3eda9420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebf3590f807c4d299f5f8e37485dc6fe)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:13.134867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84fb20-111a-475d-b00f-4ddc3eda9420 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:13.221129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ab84fb20-111a-475d-b00f-4ddc3eda9420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebf3590f807c4d299f5f8e37485dc6fe', 'request_id': 'req-ab84fb20-111a-475d-b00f-4ddc3eda9420'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:13.229108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ab84fb20-111a-475d-b00f-4ddc3eda9420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebf3590f807c4d299f5f8e37485dc6fe', 'request_id': 'req-ab84fb20-111a-475d-b00f-4ddc3eda9420'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:13.239525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ab84fb20-111a-475d-b00f-4ddc3eda9420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebf3590f807c4d299f5f8e37485dc6fe', 'request_id': 'req-ab84fb20-111a-475d-b00f-4ddc3eda9420'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:13.251135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ab84fb20-111a-475d-b00f-4ddc3eda9420 admin admin] The token cache is being invalidate because project ebf3590f807c4d299f5f8e37485dc6fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:13.251430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ab84fb20-111a-475d-b00f-4ddc3eda9420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:13.252089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ab84fb20-111a-475d-b00f-4ddc3eda9420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:13.253941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1185/2374] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:13 2022] DELETE /identity/v3/projects/ebf3590f807c4d299f5f8e37485dc6fe => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:13.258928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c7fa6-2fc0-422b-aa3a-b883bc079bd3 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:13.271296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c7fa6-2fc0-422b-aa3a-b883bc079bd3 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:13.282421 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:13.352540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420973b-977d-4dd7-9291-ed1a9d3fa683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:13.353774 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420973b-977d-4dd7-9291-ed1a9d3fa683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:13.354062 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420973b-977d-4dd7-9291-ed1a9d3fa683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:13.354353 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420973b-977d-4dd7-9291-ed1a9d3fa683 admin admin] PATH_INFO: `/v3/users/4bc6973b691846f9aad1bcb60676bf1f` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:13.355635 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420973b-977d-4dd7-9291-ed1a9d3fa683 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bc6973b691846f9aad1bcb60676bf1f)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:13.357598 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420973b-977d-4dd7-9291-ed1a9d3fa683 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:13.457083 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4420973b-977d-4dd7-9291-ed1a9d3fa683 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bc6973b691846f9aad1bcb60676bf1f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:13.477179 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4420973b-977d-4dd7-9291-ed1a9d3fa683 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bc6973b691846f9aad1bcb60676bf1f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:13.478676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4420973b-977d-4dd7-9291-ed1a9d3fa683 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bc6973b691846f9aad1bcb60676bf1f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:13.497347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4420973b-977d-4dd7-9291-ed1a9d3fa683 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bc6973b691846f9aad1bcb60676bf1f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:13.512425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4420973b-977d-4dd7-9291-ed1a9d3fa683 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bc6973b691846f9aad1bcb60676bf1f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:13.524153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1190/2375] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:13 2022] DELETE /identity/v3/users/4bc6973b691846f9aad1bcb60676bf1f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:13.905908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84fb20-111a-475d-b00f-4ddc3eda9420 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:13.907535 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ab84fb20-111a-475d-b00f-4ddc3eda9420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:13.926299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84fb20-111a-475d-b00f-4ddc3eda9420 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:13.960983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:14.002116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c01ef7-2902-4a12-a49a-f9713fbac528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:14.002811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c01ef7-2902-4a12-a49a-f9713fbac528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:14.003079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c01ef7-2902-4a12-a49a-f9713fbac528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:14.003353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c01ef7-2902-4a12-a49a-f9713fbac528 admin admin] PATH_INFO: `/v3/projects/c9ba3ac78dc543e59dea15505dfc83a5` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:14.004528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c01ef7-2902-4a12-a49a-f9713fbac528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9ba3ac78dc543e59dea15505dfc83a5)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:14.006488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c01ef7-2902-4a12-a49a-f9713fbac528 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:14.077794 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78c01ef7-2902-4a12-a49a-f9713fbac528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9ba3ac78dc543e59dea15505dfc83a5', 'request_id': 'req-78c01ef7-2902-4a12-a49a-f9713fbac528'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:14.087845 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78c01ef7-2902-4a12-a49a-f9713fbac528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9ba3ac78dc543e59dea15505dfc83a5', 'request_id': 'req-78c01ef7-2902-4a12-a49a-f9713fbac528'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:14.100806 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78c01ef7-2902-4a12-a49a-f9713fbac528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9ba3ac78dc543e59dea15505dfc83a5', 'request_id': 'req-78c01ef7-2902-4a12-a49a-f9713fbac528'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:14.116116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78c01ef7-2902-4a12-a49a-f9713fbac528 admin admin] The token cache is being invalidate because project c9ba3ac78dc543e59dea15505dfc83a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:14.116707 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78c01ef7-2902-4a12-a49a-f9713fbac528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:14.117383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78c01ef7-2902-4a12-a49a-f9713fbac528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:14.119375 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1186/2376] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:13 2022] DELETE /identity/v3/projects/c9ba3ac78dc543e59dea15505dfc83a5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:14.124165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc3fef-af65-496d-accc-036cc2c3a090 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:14.124454 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc3fef-af65-496d-accc-036cc2c3a090 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:14.124762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc3fef-af65-496d-accc-036cc2c3a090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:14.150780 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1ebc3fef-af65-496d-accc-036cc2c3a090 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:14.152051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1ebc3fef-af65-496d-accc-036cc2c3a090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:14.273028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1191/2377] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:14.283527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee60dc2-0e06-4bdc-b344-4e6ed0b6fdc8 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:14.283879 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee60dc2-0e06-4bdc-b344-4e6ed0b6fdc8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:14.284112 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee60dc2-0e06-4bdc-b344-4e6ed0b6fdc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:14.315571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3ee60dc2-0e06-4bdc-b344-4e6ed0b6fdc8 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:14.316824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3ee60dc2-0e06-4bdc-b344-4e6ed0b6fdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:14.347515 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1187/2378] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:14.352973 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc3fef-af65-496d-accc-036cc2c3a090 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:14.386743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc3fef-af65-496d-accc-036cc2c3a090 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:14.393057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2cb0e-8984-4659-be50-49f4a2f08b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:14.394177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2cb0e-8984-4659-be50-49f4a2f08b5f admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:14.394177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2cb0e-8984-4659-be50-49f4a2f08b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:14.394458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2cb0e-8984-4659-be50-49f4a2f08b5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:14.394988 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2cb0e-8984-4659-be50-49f4a2f08b5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:50:14.395286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2cb0e-8984-4659-be50-49f4a2f08b5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:14.397608 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2cb0e-8984-4659-be50-49f4a2f08b5f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:14.418459 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee60dc2-0e06-4bdc-b344-4e6ed0b6fdc8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:14.424579 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1192/2379] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:50:14.429001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee60dc2-0e06-4bdc-b344-4e6ed0b6fdc8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:14.444085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893b3ad-c769-4647-949a-355914a13027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:14.444805 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893b3ad-c769-4647-949a-355914a13027 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:14.445198 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893b3ad-c769-4647-949a-355914a13027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:14.445566 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893b3ad-c769-4647-949a-355914a13027 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:14.447087 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893b3ad-c769-4647-949a-355914a13027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:14.449610 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893b3ad-c769-4647-949a-355914a13027 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:14.503041 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1188/2380] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:14 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:14.515575 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2cb0e-8984-4659-be50-49f4a2f08b5f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:14.552561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2cb0e-8984-4659-be50-49f4a2f08b5f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:14.574578 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:14.662339 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c28f30-6c62-4a9f-a26f-a5f945011410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:14.663031 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c28f30-6c62-4a9f-a26f-a5f945011410 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:14.663330 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c28f30-6c62-4a9f-a26f-a5f945011410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:14.663549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c28f30-6c62-4a9f-a26f-a5f945011410 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:14.664238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c28f30-6c62-4a9f-a26f-a5f945011410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:14.666800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c28f30-6c62-4a9f-a26f-a5f945011410 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:14.700694 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1193/2381] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:14 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:14.704184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893b3ad-c769-4647-949a-355914a13027 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:14.719709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893b3ad-c769-4647-949a-355914a13027 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:14.725703 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f782d6-327b-436f-b92c-ef3cd4e00585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:14.726732 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f782d6-327b-436f-b92c-ef3cd4e00585 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:14.727224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f782d6-327b-436f-b92c-ef3cd4e00585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:14.728529 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f782d6-327b-436f-b92c-ef3cd4e00585 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:14.729507 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f782d6-327b-436f-b92c-ef3cd4e00585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:14.741768 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f782d6-327b-436f-b92c-ef3cd4e00585 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:14.762619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1189/2382] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:14.763843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c28f30-6c62-4a9f-a26f-a5f945011410 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:14.784615 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c28f30-6c62-4a9f-a26f-a5f945011410 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:14.790956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3fe7f-7d52-4a63-b3b5-8c0810bedfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:14.791276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3fe7f-7d52-4a63-b3b5-8c0810bedfeb admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:14.791568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3fe7f-7d52-4a63-b3b5-8c0810bedfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:14.791845 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3fe7f-7d52-4a63-b3b5-8c0810bedfeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:14.792556 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3fe7f-7d52-4a63-b3b5-8c0810bedfeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:14.794214 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3fe7f-7d52-4a63-b3b5-8c0810bedfeb admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:14.821575 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1194/2383] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:14.823197 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f782d6-327b-436f-b92c-ef3cd4e00585 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:14.845428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f782d6-327b-436f-b92c-ef3cd4e00585 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:14.856910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df7f7c-d601-4396-93d6-228bc4c8b542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:14.859525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df7f7c-d601-4396-93d6-228bc4c8b542 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:14.859525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df7f7c-d601-4396-93d6-228bc4c8b542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:14.859525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df7f7c-d601-4396-93d6-228bc4c8b542 admin admin] PATH_INFO: `/v3/projects/02423bb1648a49b58d82be27e7289657/users/9ad9562baa964b6c95e861be3aa021e5/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:14.896812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df7f7c-d601-4396-93d6-228bc4c8b542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02423bb1648a49b58d82be27e7289657, user_id=9ad9562baa964b6c95e861be3aa021e5, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:14.900688 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df7f7c-d601-4396-93d6-228bc4c8b542 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:14.925699 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1190/2384] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:14 2022] PUT /identity/v3/projects/02423bb1648a49b58d82be27e7289657/users/9ad9562baa964b6c95e861be3aa021e5/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:14.931085 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cacbdc-2f9a-4544-a62d-2116d1431ef3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:14.931378 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cacbdc-2f9a-4544-a62d-2116d1431ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:14.933723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cacbdc-2f9a-4544-a62d-2116d1431ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:14.975297 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-99cacbdc-2f9a-4544-a62d-2116d1431ef3 None None] MFA Rules not processed for user `9ad9562baa964b6c95e861be3aa021e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:14.976689 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-99cacbdc-2f9a-4544-a62d-2116d1431ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:15.080360 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1195/2385] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:15.085530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df7f7c-d601-4396-93d6-228bc4c8b542 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:15.190166 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:15.203005 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df7f7c-d601-4396-93d6-228bc4c8b542 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:15.217709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d8d3e-4db7-47c1-867c-1d699b9c8eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:15.218359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d8d3e-4db7-47c1-867c-1d699b9c8eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:15.218645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d8d3e-4db7-47c1-867c-1d699b9c8eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:15.218922 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d8d3e-4db7-47c1-867c-1d699b9c8eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:15.219615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d8d3e-4db7-47c1-867c-1d699b9c8eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:15.221803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d8d3e-4db7-47c1-867c-1d699b9c8eb7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:15.278721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cacbdc-2f9a-4544-a62d-2116d1431ef3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:15.281675 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1191/2386] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:15 2022] POST /identity/v3/projects => generated 375 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:15.298742 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cacbdc-2f9a-4544-a62d-2116d1431ef3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:15.406738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f2fd0-adaf-4d43-9e04-ca56a52a6699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:15.407419 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f2fd0-adaf-4d43-9e04-ca56a52a6699 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:15.407742 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f2fd0-adaf-4d43-9e04-ca56a52a6699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:15.408031 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f2fd0-adaf-4d43-9e04-ca56a52a6699 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:15.409203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f2fd0-adaf-4d43-9e04-ca56a52a6699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:15.411458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f2fd0-adaf-4d43-9e04-ca56a52a6699 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:15.458681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d8d3e-4db7-47c1-867c-1d699b9c8eb7 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:15.459847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1196/2387] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:15 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:15.480106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d8d3e-4db7-47c1-867c-1d699b9c8eb7 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:15.493453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63e621-f865-4b34-a27d-c0ace93b5ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:15.497700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63e621-f865-4b34-a27d-c0ace93b5ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:15.499145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63e621-f865-4b34-a27d-c0ace93b5ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:15.499763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63e621-f865-4b34-a27d-c0ace93b5ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:15.502139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63e621-f865-4b34-a27d-c0ace93b5ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:15.503870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63e621-f865-4b34-a27d-c0ace93b5ad2 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:15.531905 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1192/2388] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:50:15.536198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f2fd0-adaf-4d43-9e04-ca56a52a6699 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:15.559451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f2fd0-adaf-4d43-9e04-ca56a52a6699 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:15.589180 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05686334-e148-4674-850e-420a52e92fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:15.589915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05686334-e148-4674-850e-420a52e92fde admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:15.590194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05686334-e148-4674-850e-420a52e92fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:15.590465 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05686334-e148-4674-850e-420a52e92fde admin admin] PATH_INFO: `/v3/projects/7dd3370eeb734ed7aab4cb03350dd2a4/users/2397d6a8ca044926b7bdeaeb65a959f0/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:15.635428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:15.651576 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05686334-e148-4674-850e-420a52e92fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dd3370eeb734ed7aab4cb03350dd2a4, user_id=2397d6a8ca044926b7bdeaeb65a959f0, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:15.654271 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05686334-e148-4674-850e-420a52e92fde admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:15.687478 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1197/2389] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:15 2022] PUT /identity/v3/projects/7dd3370eeb734ed7aab4cb03350dd2a4/users/2397d6a8ca044926b7bdeaeb65a959f0/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:50:15.692786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63e621-f865-4b34-a27d-c0ace93b5ad2 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:15.736874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63e621-f865-4b34-a27d-c0ace93b5ad2 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:15.888876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9920e0-b9db-4a54-9f6c-bd1bfabe3d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:15.890663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9920e0-b9db-4a54-9f6c-bd1bfabe3d93 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:15.891237 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9920e0-b9db-4a54-9f6c-bd1bfabe3d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:15.891732 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9920e0-b9db-4a54-9f6c-bd1bfabe3d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:15.892762 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9920e0-b9db-4a54-9f6c-bd1bfabe3d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:15.895034 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9920e0-b9db-4a54-9f6c-bd1bfabe3d93 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:15.912433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1193/2390] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:15.919874 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05686334-e148-4674-850e-420a52e92fde admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:15.986041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05686334-e148-4674-850e-420a52e92fde admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:15.992781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676cab6-1395-46f2-bb47-52b8601e1402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:15.995059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676cab6-1395-46f2-bb47-52b8601e1402 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:15.995570 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676cab6-1395-46f2-bb47-52b8601e1402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:15.995946 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676cab6-1395-46f2-bb47-52b8601e1402 admin admin] PATH_INFO: `/v3/projects/7dd3370eeb734ed7aab4cb03350dd2a4/users/2397d6a8ca044926b7bdeaeb65a959f0/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:15.998337 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676cab6-1395-46f2-bb47-52b8601e1402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dd3370eeb734ed7aab4cb03350dd2a4, user_id=2397d6a8ca044926b7bdeaeb65a959f0, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:16.001757 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676cab6-1395-46f2-bb47-52b8601e1402 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:16.051115 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1198/2391] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:15 2022] PUT /identity/v3/projects/7dd3370eeb734ed7aab4cb03350dd2a4/users/2397d6a8ca044926b7bdeaeb65a959f0/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:16.065556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384027d-55a7-4e73-a14b-25a98eedf27f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:16.066130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384027d-55a7-4e73-a14b-25a98eedf27f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:16.066699 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384027d-55a7-4e73-a14b-25a98eedf27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:16.129796 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a384027d-55a7-4e73-a14b-25a98eedf27f None None] MFA Rules not processed for user `2397d6a8ca044926b7bdeaeb65a959f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:16.132419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a384027d-55a7-4e73-a14b-25a98eedf27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:16.258912 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1194/2392] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:16.272912 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676cab6-1395-46f2-bb47-52b8601e1402 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:16.277404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b676cab6-1395-46f2-bb47-52b8601e1402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:16.362604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676cab6-1395-46f2-bb47-52b8601e1402 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:16.367081 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b8a75-036e-41cf-b0d9-3abfc803b883 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:16.367717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b8a75-036e-41cf-b0d9-3abfc803b883 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:16.368053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b8a75-036e-41cf-b0d9-3abfc803b883 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:16.368672 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b8a75-036e-41cf-b0d9-3abfc803b883 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:16.452646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b8a75-036e-41cf-b0d9-3abfc803b883 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:16.456121 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b8a75-036e-41cf-b0d9-3abfc803b883 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:16.477137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1199/2393] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:19.460693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384027d-55a7-4e73-a14b-25a98eedf27f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:19.520306 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:19.533139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384027d-55a7-4e73-a14b-25a98eedf27f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:19.537825 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612b5fe-7b4e-450b-adde-dd41b7c089bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:19.538496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612b5fe-7b4e-450b-adde-dd41b7c089bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:19.538780 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612b5fe-7b4e-450b-adde-dd41b7c089bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:19.539049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612b5fe-7b4e-450b-adde-dd41b7c089bc admin admin] PATH_INFO: `/v3/users/9ad9562baa964b6c95e861be3aa021e5` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:19.540743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612b5fe-7b4e-450b-adde-dd41b7c089bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ad9562baa964b6c95e861be3aa021e5)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:19.542712 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612b5fe-7b4e-450b-adde-dd41b7c089bc admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:19.610803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6612b5fe-7b4e-450b-adde-dd41b7c089bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ad9562baa964b6c95e861be3aa021e5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:19.623112 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6612b5fe-7b4e-450b-adde-dd41b7c089bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ad9562baa964b6c95e861be3aa021e5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:19.633872 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6612b5fe-7b4e-450b-adde-dd41b7c089bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ad9562baa964b6c95e861be3aa021e5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:19.634679 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6612b5fe-7b4e-450b-adde-dd41b7c089bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ad9562baa964b6c95e861be3aa021e5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:19.635743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6612b5fe-7b4e-450b-adde-dd41b7c089bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ad9562baa964b6c95e861be3aa021e5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:19.674724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1195/2394] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:19 2022] DELETE /identity/v3/users/9ad9562baa964b6c95e861be3aa021e5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:19.679140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5359126-d676-407f-871e-aede090553bb None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:19.679630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5359126-d676-407f-871e-aede090553bb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:19.679805 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5359126-d676-407f-871e-aede090553bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:19.706025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a5359126-d676-407f-871e-aede090553bb None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:19.708634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a5359126-d676-407f-871e-aede090553bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:19.819194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1200/2395] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:19.830240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612b5fe-7b4e-450b-adde-dd41b7c089bc admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:19.831790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6612b5fe-7b4e-450b-adde-dd41b7c089bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:19.901731 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612b5fe-7b4e-450b-adde-dd41b7c089bc admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:19.906003 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46728b-3e22-4de6-a9ab-9f1efa2e55a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:19.907194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46728b-3e22-4de6-a9ab-9f1efa2e55a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:19.907194 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46728b-3e22-4de6-a9ab-9f1efa2e55a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:19.907702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46728b-3e22-4de6-a9ab-9f1efa2e55a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:19.919264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46728b-3e22-4de6-a9ab-9f1efa2e55a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:19.920734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46728b-3e22-4de6-a9ab-9f1efa2e55a6 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:19.932814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1196/2396] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:20.351793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5359126-d676-407f-871e-aede090553bb None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:20.353336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a5359126-d676-407f-871e-aede090553bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:20.368156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5359126-d676-407f-871e-aede090553bb None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:20.372967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919ab2f-d089-4d7e-9b72-84611dab8411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:20.373660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919ab2f-d089-4d7e-9b72-84611dab8411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:20.373939 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919ab2f-d089-4d7e-9b72-84611dab8411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:20.374209 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919ab2f-d089-4d7e-9b72-84611dab8411 admin admin] PATH_INFO: `/v3/projects/02423bb1648a49b58d82be27e7289657` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:20.375398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919ab2f-d089-4d7e-9b72-84611dab8411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02423bb1648a49b58d82be27e7289657)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:20.377861 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919ab2f-d089-4d7e-9b72-84611dab8411 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:20.414205 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:20.433073 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b919ab2f-d089-4d7e-9b72-84611dab8411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02423bb1648a49b58d82be27e7289657', 'request_id': 'req-b919ab2f-d089-4d7e-9b72-84611dab8411'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:20.441418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b919ab2f-d089-4d7e-9b72-84611dab8411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02423bb1648a49b58d82be27e7289657', 'request_id': 'req-b919ab2f-d089-4d7e-9b72-84611dab8411'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:20.448287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b919ab2f-d089-4d7e-9b72-84611dab8411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02423bb1648a49b58d82be27e7289657', 'request_id': 'req-b919ab2f-d089-4d7e-9b72-84611dab8411'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:20.458928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b919ab2f-d089-4d7e-9b72-84611dab8411 admin admin] The token cache is being invalidate because project 02423bb1648a49b58d82be27e7289657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:20.459248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b919ab2f-d089-4d7e-9b72-84611dab8411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:20.459900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b919ab2f-d089-4d7e-9b72-84611dab8411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:20.461436 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1201/2397] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:20 2022] DELETE /identity/v3/projects/02423bb1648a49b58d82be27e7289657 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:20.465854 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46728b-3e22-4de6-a9ab-9f1efa2e55a6 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:20.475676 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46728b-3e22-4de6-a9ab-9f1efa2e55a6 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:20.538963 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233af7f-2e1d-4aa7-b0b8-0133dd671112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:20.540338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233af7f-2e1d-4aa7-b0b8-0133dd671112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:20.540848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233af7f-2e1d-4aa7-b0b8-0133dd671112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:20.541371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233af7f-2e1d-4aa7-b0b8-0133dd671112 admin admin] PATH_INFO: `/v3/users/2397d6a8ca044926b7bdeaeb65a959f0` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:20.542948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233af7f-2e1d-4aa7-b0b8-0133dd671112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2397d6a8ca044926b7bdeaeb65a959f0)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:20.545437 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233af7f-2e1d-4aa7-b0b8-0133dd671112 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:20.561648 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:20.599461 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8233af7f-2e1d-4aa7-b0b8-0133dd671112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2397d6a8ca044926b7bdeaeb65a959f0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:20.607675 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8233af7f-2e1d-4aa7-b0b8-0133dd671112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2397d6a8ca044926b7bdeaeb65a959f0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:20.615447 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8233af7f-2e1d-4aa7-b0b8-0133dd671112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2397d6a8ca044926b7bdeaeb65a959f0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:20.616175 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8233af7f-2e1d-4aa7-b0b8-0133dd671112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2397d6a8ca044926b7bdeaeb65a959f0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:20.616776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8233af7f-2e1d-4aa7-b0b8-0133dd671112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2397d6a8ca044926b7bdeaeb65a959f0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:20.636812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1197/2398] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:20 2022] DELETE /identity/v3/users/2397d6a8ca044926b7bdeaeb65a959f0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:20.952654 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919ab2f-d089-4d7e-9b72-84611dab8411 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:20.954832 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b919ab2f-d089-4d7e-9b72-84611dab8411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:20.970232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919ab2f-d089-4d7e-9b72-84611dab8411 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:21.053623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bdd40-0f03-4902-9e88-0cdb3727158b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:21.054428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bdd40-0f03-4902-9e88-0cdb3727158b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:21.057582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bdd40-0f03-4902-9e88-0cdb3727158b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:21.057582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bdd40-0f03-4902-9e88-0cdb3727158b admin admin] PATH_INFO: `/v3/projects/7dd3370eeb734ed7aab4cb03350dd2a4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:21.057582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22bdd40-0f03-4902-9e88-0cdb3727158b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dd3370eeb734ed7aab4cb03350dd2a4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:21.059696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22bdd40-0f03-4902-9e88-0cdb3727158b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:21.130995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f22bdd40-0f03-4902-9e88-0cdb3727158b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dd3370eeb734ed7aab4cb03350dd2a4', 'request_id': 'req-f22bdd40-0f03-4902-9e88-0cdb3727158b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:21.140248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f22bdd40-0f03-4902-9e88-0cdb3727158b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dd3370eeb734ed7aab4cb03350dd2a4', 'request_id': 'req-f22bdd40-0f03-4902-9e88-0cdb3727158b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:21.148692 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f22bdd40-0f03-4902-9e88-0cdb3727158b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dd3370eeb734ed7aab4cb03350dd2a4', 'request_id': 'req-f22bdd40-0f03-4902-9e88-0cdb3727158b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:21.159656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f22bdd40-0f03-4902-9e88-0cdb3727158b admin admin] The token cache is being invalidate because project 7dd3370eeb734ed7aab4cb03350dd2a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:21.159944 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f22bdd40-0f03-4902-9e88-0cdb3727158b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:21.160623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f22bdd40-0f03-4902-9e88-0cdb3727158b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:21.162210 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1202/2399] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:20 2022] DELETE /identity/v3/projects/7dd3370eeb734ed7aab4cb03350dd2a4 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:21.167247 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e28f78-8608-414d-8b30-1994de4aa3df None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:21.167535 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e28f78-8608-414d-8b30-1994de4aa3df None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:21.167813 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e28f78-8608-414d-8b30-1994de4aa3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:21.187677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a8e28f78-8608-414d-8b30-1994de4aa3df None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:21.188991 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a8e28f78-8608-414d-8b30-1994de4aa3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:21.256677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1198/2400] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:21.265848 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acfae3-cf54-45ab-9bfe-74e8c9b84a29 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:21.266184 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acfae3-cf54-45ab-9bfe-74e8c9b84a29 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:21.266503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acfae3-cf54-45ab-9bfe-74e8c9b84a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:21.291049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e2acfae3-cf54-45ab-9bfe-74e8c9b84a29 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:21.292282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e2acfae3-cf54-45ab-9bfe-74e8c9b84a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:21.302022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1203/2401] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:21.306081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e28f78-8608-414d-8b30-1994de4aa3df None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:21.317187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e28f78-8608-414d-8b30-1994de4aa3df None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:21.323929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60476eb6-e408-4446-a1bb-bd99af06aaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:21.324681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60476eb6-e408-4446-a1bb-bd99af06aaef admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:21.326841 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60476eb6-e408-4446-a1bb-bd99af06aaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:21.327292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60476eb6-e408-4446-a1bb-bd99af06aaef admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:21.328157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60476eb6-e408-4446-a1bb-bd99af06aaef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:50:21.328560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60476eb6-e408-4446-a1bb-bd99af06aaef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:21.330325 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60476eb6-e408-4446-a1bb-bd99af06aaef admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:21.349506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1199/2402] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:50:21.356272 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acfae3-cf54-45ab-9bfe-74e8c9b84a29 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:21.366547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acfae3-cf54-45ab-9bfe-74e8c9b84a29 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:21.371413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df3506-f0bb-41d4-886a-0c9d489d6b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:21.372041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df3506-f0bb-41d4-886a-0c9d489d6b85 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:21.372323 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df3506-f0bb-41d4-886a-0c9d489d6b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:21.372643 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df3506-f0bb-41d4-886a-0c9d489d6b85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:21.373324 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df3506-f0bb-41d4-886a-0c9d489d6b85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:21.375428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df3506-f0bb-41d4-886a-0c9d489d6b85 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:21.409508 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1204/2403] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:21 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:21.414462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60476eb6-e408-4446-a1bb-bd99af06aaef admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:21.425757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60476eb6-e408-4446-a1bb-bd99af06aaef admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:21.494971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f42a5-49bf-4a71-8553-5cde46ee7a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:21.495929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f42a5-49bf-4a71-8553-5cde46ee7a91 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:21.495929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f42a5-49bf-4a71-8553-5cde46ee7a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:21.496466 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f42a5-49bf-4a71-8553-5cde46ee7a91 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:21.496845 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f42a5-49bf-4a71-8553-5cde46ee7a91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:21.498987 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f42a5-49bf-4a71-8553-5cde46ee7a91 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:21.523643 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1200/2404] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:21 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:21.528388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df3506-f0bb-41d4-886a-0c9d489d6b85 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:21.539498 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df3506-f0bb-41d4-886a-0c9d489d6b85 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:21.545589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a56644-1005-4a71-a460-15601130f1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:21.546306 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a56644-1005-4a71-a460-15601130f1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:21.546584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a56644-1005-4a71-a460-15601130f1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:21.546855 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a56644-1005-4a71-a460-15601130f1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:21.547560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a56644-1005-4a71-a460-15601130f1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:21.549169 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a56644-1005-4a71-a460-15601130f1a9 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:21.561983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1205/2405] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:50:21.567069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f42a5-49bf-4a71-8553-5cde46ee7a91 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:21.579591 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f42a5-49bf-4a71-8553-5cde46ee7a91 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:21.584717 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b3fc9-0c40-4c95-ae1f-8580c7ca7e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:21.585349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b3fc9-0c40-4c95-ae1f-8580c7ca7e2a admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:21.585696 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b3fc9-0c40-4c95-ae1f-8580c7ca7e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:21.585973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b3fc9-0c40-4c95-ae1f-8580c7ca7e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:21.587381 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b3fc9-0c40-4c95-ae1f-8580c7ca7e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:21.588988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b3fc9-0c40-4c95-ae1f-8580c7ca7e2a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:21.591744 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:21.603907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1201/2406] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:21.608853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a56644-1005-4a71-a460-15601130f1a9 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:21.619525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a56644-1005-4a71-a460-15601130f1a9 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:21.624411 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806f68d-37cf-41c2-a66e-1b31d6227993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:21.625089 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806f68d-37cf-41c2-a66e-1b31d6227993 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:21.625367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806f68d-37cf-41c2-a66e-1b31d6227993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:21.625671 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806f68d-37cf-41c2-a66e-1b31d6227993 admin admin] PATH_INFO: `/v3/projects/530f840de11e4324a99e9b5161d2a3f8/users/54bbc7809d024326901ca92bb7c04564/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:21.628244 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:21.646921 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c806f68d-37cf-41c2-a66e-1b31d6227993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=530f840de11e4324a99e9b5161d2a3f8, user_id=54bbc7809d024326901ca92bb7c04564, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:21.650444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c806f68d-37cf-41c2-a66e-1b31d6227993 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:21.663837 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1206/2407] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:21 2022] PUT /identity/v3/projects/530f840de11e4324a99e9b5161d2a3f8/users/54bbc7809d024326901ca92bb7c04564/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:21.677392 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d687262-9464-4fc3-87e6-ef2f04cf6870 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:21.677858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d687262-9464-4fc3-87e6-ef2f04cf6870 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:21.678253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d687262-9464-4fc3-87e6-ef2f04cf6870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:21.711640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4d687262-9464-4fc3-87e6-ef2f04cf6870 None None] MFA Rules not processed for user `54bbc7809d024326901ca92bb7c04564`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:21.713395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4d687262-9464-4fc3-87e6-ef2f04cf6870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:21.782232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1202/2408] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:21.793760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806f68d-37cf-41c2-a66e-1b31d6227993 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:21.795734 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c806f68d-37cf-41c2-a66e-1b31d6227993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:21.855753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806f68d-37cf-41c2-a66e-1b31d6227993 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:21.859537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642cef5-7dec-4bac-b83d-63189eccae72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:21.860162 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642cef5-7dec-4bac-b83d-63189eccae72 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:21.860450 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642cef5-7dec-4bac-b83d-63189eccae72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:21.860728 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642cef5-7dec-4bac-b83d-63189eccae72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:21.870252 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642cef5-7dec-4bac-b83d-63189eccae72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:21.871717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642cef5-7dec-4bac-b83d-63189eccae72 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:21.881459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1207/2409] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:22.238439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d687262-9464-4fc3-87e6-ef2f04cf6870 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:22.246573 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4d687262-9464-4fc3-87e6-ef2f04cf6870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:22.340790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d687262-9464-4fc3-87e6-ef2f04cf6870 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:22.346772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83b4d8-1fcb-42d9-8bc3-e5e896b362bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:22.347321 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83b4d8-1fcb-42d9-8bc3-e5e896b362bb service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:22.347597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83b4d8-1fcb-42d9-8bc3-e5e896b362bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:22.347866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83b4d8-1fcb-42d9-8bc3-e5e896b362bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:22.349836 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5e83b4d8-1fcb-42d9-8bc3-e5e896b362bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:22.428908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83b4d8-1fcb-42d9-8bc3-e5e896b362bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:22.430299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83b4d8-1fcb-42d9-8bc3-e5e896b362bb service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:22.442180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1203/2410] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:28.207041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d540f6-83aa-425a-9304-ba1ea110129a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:28.207923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d540f6-83aa-425a-9304-ba1ea110129a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:28.208385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d540f6-83aa-425a-9304-ba1ea110129a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:28.237195 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a9d540f6-83aa-425a-9304-ba1ea110129a None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:28.238797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a9d540f6-83aa-425a-9304-ba1ea110129a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:28.321553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1208/2411] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 3 10:50:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:28.334353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83b4d8-1fcb-42d9-8bc3-e5e896b362bb service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:28.335875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5e83b4d8-1fcb-42d9-8bc3-e5e896b362bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:28.396971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:28.406783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d540f6-83aa-425a-9304-ba1ea110129a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:28.408567 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a9d540f6-83aa-425a-9304-ba1ea110129a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:28.433407 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d540f6-83aa-425a-9304-ba1ea110129a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:28.438132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338879b-cbc8-41a6-9fa7-95447f7447a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:28.438774 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338879b-cbc8-41a6-9fa7-95447f7447a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:28.439064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338879b-cbc8-41a6-9fa7-95447f7447a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:28.439324 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338879b-cbc8-41a6-9fa7-95447f7447a6 admin admin] PATH_INFO: `/v3/users/ebdb81686d804eb3864b97481643fc2b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:28.440502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338879b-cbc8-41a6-9fa7-95447f7447a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebdb81686d804eb3864b97481643fc2b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:28.442459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338879b-cbc8-41a6-9fa7-95447f7447a6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:28.443289 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83b4d8-1fcb-42d9-8bc3-e5e896b362bb service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:28.449288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857935cf-d68e-4768-a898-821aaca860ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:28.450037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857935cf-d68e-4768-a898-821aaca860ef service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:28.450385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857935cf-d68e-4768-a898-821aaca860ef service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:28.450721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857935cf-d68e-4768-a898-821aaca860ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:28.460673 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857935cf-d68e-4768-a898-821aaca860ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:28.465365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857935cf-d68e-4768-a898-821aaca860ef service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:28.475561 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1204/2412] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:50:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 10:50:28.502228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f338879b-cbc8-41a6-9fa7-95447f7447a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebdb81686d804eb3864b97481643fc2b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:28.529708 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f338879b-cbc8-41a6-9fa7-95447f7447a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebdb81686d804eb3864b97481643fc2b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:28.530603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f338879b-cbc8-41a6-9fa7-95447f7447a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebdb81686d804eb3864b97481643fc2b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:28.544582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f338879b-cbc8-41a6-9fa7-95447f7447a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebdb81686d804eb3864b97481643fc2b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:28.583207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f338879b-cbc8-41a6-9fa7-95447f7447a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebdb81686d804eb3864b97481643fc2b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:28.596723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1209/2413] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:28 2022] DELETE /identity/v3/users/ebdb81686d804eb3864b97481643fc2b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:50:28.934453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857935cf-d68e-4768-a898-821aaca860ef service swift] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:28.948064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857935cf-d68e-4768-a898-821aaca860ef service swift] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:29.112461 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00e9db-a563-491b-b846-00c33cfb58c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:29.115632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00e9db-a563-491b-b846-00c33cfb58c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:29.118987 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00e9db-a563-491b-b846-00c33cfb58c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:29.119799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00e9db-a563-491b-b846-00c33cfb58c6 admin admin] PATH_INFO: `/v3/projects/bcfc3e09588c48299fdf5634cda57416` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:29.120637 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00e9db-a563-491b-b846-00c33cfb58c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcfc3e09588c48299fdf5634cda57416)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:29.122804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00e9db-a563-491b-b846-00c33cfb58c6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:29.212142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-cd00e9db-a563-491b-b846-00c33cfb58c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcfc3e09588c48299fdf5634cda57416', 'request_id': 'req-cd00e9db-a563-491b-b846-00c33cfb58c6'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:29.221566 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-cd00e9db-a563-491b-b846-00c33cfb58c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcfc3e09588c48299fdf5634cda57416', 'request_id': 'req-cd00e9db-a563-491b-b846-00c33cfb58c6'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:29.234904 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-cd00e9db-a563-491b-b846-00c33cfb58c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcfc3e09588c48299fdf5634cda57416', 'request_id': 'req-cd00e9db-a563-491b-b846-00c33cfb58c6'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:29.256094 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-cd00e9db-a563-491b-b846-00c33cfb58c6 admin admin] The token cache is being invalidate because project bcfc3e09588c48299fdf5634cda57416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:29.256516 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-cd00e9db-a563-491b-b846-00c33cfb58c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:29.257289 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-cd00e9db-a563-491b-b846-00c33cfb58c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:29.260382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1205/2414] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:28 2022] DELETE /identity/v3/projects/bcfc3e09588c48299fdf5634cda57416 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:29.264858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe89d35-0ac6-46e3-99c9-878f78fc3d0c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:29.265428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe89d35-0ac6-46e3-99c9-878f78fc3d0c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:29.265428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe89d35-0ac6-46e3-99c9-878f78fc3d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:29.281345 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:29.326737 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4fe89d35-0ac6-46e3-99c9-878f78fc3d0c None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:29.329404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4fe89d35-0ac6-46e3-99c9-878f78fc3d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:29.508194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1210/2415] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:29.517881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7473c46-fb75-48e3-99a6-88532e3f860a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:29.518453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7473c46-fb75-48e3-99a6-88532e3f860a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:29.518791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7473c46-fb75-48e3-99a6-88532e3f860a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:29.579638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b7473c46-fb75-48e3-99a6-88532e3f860a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:29.581287 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b7473c46-fb75-48e3-99a6-88532e3f860a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:29.592334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1206/2416] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:29.597772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe89d35-0ac6-46e3-99c9-878f78fc3d0c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:29.618944 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe89d35-0ac6-46e3-99c9-878f78fc3d0c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:29.629876 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d967105-fb79-4aba-9c81-48b13aac06f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:29.631241 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d967105-fb79-4aba-9c81-48b13aac06f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:29.631755 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d967105-fb79-4aba-9c81-48b13aac06f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:29.632382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d967105-fb79-4aba-9c81-48b13aac06f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:29.633352 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d967105-fb79-4aba-9c81-48b13aac06f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:50:29.634405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d967105-fb79-4aba-9c81-48b13aac06f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:29.638334 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d967105-fb79-4aba-9c81-48b13aac06f3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:29.661263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1211/2417] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:50:29.666084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7473c46-fb75-48e3-99a6-88532e3f860a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:29.685599 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7473c46-fb75-48e3-99a6-88532e3f860a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:29.693310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d39c3a-4121-4737-a0ca-774289b6b0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:29.695900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d39c3a-4121-4737-a0ca-774289b6b0a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:29.697277 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d39c3a-4121-4737-a0ca-774289b6b0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:29.697764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d39c3a-4121-4737-a0ca-774289b6b0a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:29.698750 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d39c3a-4121-4737-a0ca-774289b6b0a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:29.701875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d39c3a-4121-4737-a0ca-774289b6b0a7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:29.764753 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1207/2418] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:29 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:29.769367 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d967105-fb79-4aba-9c81-48b13aac06f3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:29.784588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d967105-fb79-4aba-9c81-48b13aac06f3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:29.908496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3520b62-9df7-4663-b430-7ef69f747a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:29.909070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3520b62-9df7-4663-b430-7ef69f747a03 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:29.909368 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3520b62-9df7-4663-b430-7ef69f747a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:29.909722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3520b62-9df7-4663-b430-7ef69f747a03 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:29.910336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3520b62-9df7-4663-b430-7ef69f747a03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:29.912686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3520b62-9df7-4663-b430-7ef69f747a03 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:29.968587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d39c3a-4121-4737-a0ca-774289b6b0a7 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:29.979131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1212/2419] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:29 2022] POST /identity/v3/users => generated 302 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:29.982956 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d39c3a-4121-4737-a0ca-774289b6b0a7 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:30.000476 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ef842-2f7a-4638-a5be-fa88eb05d176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:30.001126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ef842-2f7a-4638-a5be-fa88eb05d176 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:30.001410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ef842-2f7a-4638-a5be-fa88eb05d176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:30.001714 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ef842-2f7a-4638-a5be-fa88eb05d176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:30.002420 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ef842-2f7a-4638-a5be-fa88eb05d176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:30.004050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ef842-2f7a-4638-a5be-fa88eb05d176 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:30.026478 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1208/2420] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:50:30.033240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3520b62-9df7-4663-b430-7ef69f747a03 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:30.058939 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3520b62-9df7-4663-b430-7ef69f747a03 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:30.083914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebec3f-86c3-40b0-9db1-f5c2418dc975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:30.084573 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebec3f-86c3-40b0-9db1-f5c2418dc975 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:30.084863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebec3f-86c3-40b0-9db1-f5c2418dc975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:30.085137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebec3f-86c3-40b0-9db1-f5c2418dc975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:30.086014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebec3f-86c3-40b0-9db1-f5c2418dc975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:30.087929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebec3f-86c3-40b0-9db1-f5c2418dc975 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:30.106757 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1213/2421] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:30.111199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ef842-2f7a-4638-a5be-fa88eb05d176 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:30.127437 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ef842-2f7a-4638-a5be-fa88eb05d176 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:30.136031 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf3571-0a95-45d2-a0c1-6cd91308ccba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:30.136726 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf3571-0a95-45d2-a0c1-6cd91308ccba admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:30.137014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf3571-0a95-45d2-a0c1-6cd91308ccba admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:30.137289 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf3571-0a95-45d2-a0c1-6cd91308ccba admin admin] PATH_INFO: `/v3/projects/a0163ced4c2f49c09b1a7e17814fb6c1/users/6d87144bec4c437cbb24335ce7726f17/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:30.172836 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf3571-0a95-45d2-a0c1-6cd91308ccba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0163ced4c2f49c09b1a7e17814fb6c1, user_id=6d87144bec4c437cbb24335ce7726f17, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:30.175646 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf3571-0a95-45d2-a0c1-6cd91308ccba admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:30.178557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:30.214346 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1209/2422] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:30 2022] PUT /identity/v3/projects/a0163ced4c2f49c09b1a7e17814fb6c1/users/6d87144bec4c437cbb24335ce7726f17/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:50:30.226927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2484c64-a7ee-40f4-a3e3-773ed2a7228d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:30.227789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2484c64-a7ee-40f4-a3e3-773ed2a7228d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:30.228035 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2484c64-a7ee-40f4-a3e3-773ed2a7228d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:30.248390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e2484c64-a7ee-40f4-a3e3-773ed2a7228d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:30.249681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e2484c64-a7ee-40f4-a3e3-773ed2a7228d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:30.410603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1214/2423] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:30.425223 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf3571-0a95-45d2-a0c1-6cd91308ccba admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:30.427461 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-31bf3571-0a95-45d2-a0c1-6cd91308ccba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:30.542958 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf3571-0a95-45d2-a0c1-6cd91308ccba admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:30.547881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b988fc-eeb6-4021-b4d7-9b68882525b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:30.548760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b988fc-eeb6-4021-b4d7-9b68882525b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:30.549202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b988fc-eeb6-4021-b4d7-9b68882525b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:30.549665 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b988fc-eeb6-4021-b4d7-9b68882525b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:30.561755 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b988fc-eeb6-4021-b4d7-9b68882525b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:30.563568 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b988fc-eeb6-4021-b4d7-9b68882525b7 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:30.577730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1210/2424] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:31.511973 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdb04f-2470-4058-80b4-2d35409d52c3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:31.512505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdb04f-2470-4058-80b4-2d35409d52c3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:31.512505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdb04f-2470-4058-80b4-2d35409d52c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:31.539346 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:31.554106 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-adcdb04f-2470-4058-80b4-2d35409d52c3 None None] MFA Rules not processed for user `6d87144bec4c437cbb24335ce7726f17`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:31.557665 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-adcdb04f-2470-4058-80b4-2d35409d52c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:31.718213 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1215/2425] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:31 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:31.723225 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b988fc-eeb6-4021-b4d7-9b68882525b7 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:31.749154 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b988fc-eeb6-4021-b4d7-9b68882525b7 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:31.762866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5b5bf-deec-4ae6-b0a0-cae4bd1620a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:31.763605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5b5bf-deec-4ae6-b0a0-cae4bd1620a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:31.764198 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5b5bf-deec-4ae6-b0a0-cae4bd1620a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:31.764729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5b5bf-deec-4ae6-b0a0-cae4bd1620a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:31.765744 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5b5bf-deec-4ae6-b0a0-cae4bd1620a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:31.768689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5b5bf-deec-4ae6-b0a0-cae4bd1620a8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:31.815580 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1211/2426] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:31 2022] POST /identity/v3/projects => generated 401 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:31.819623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdb04f-2470-4058-80b4-2d35409d52c3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:31.830910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdb04f-2470-4058-80b4-2d35409d52c3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:31.919915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd694998-bd69-4bd5-a6ae-0857c0d4eb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:31.920640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd694998-bd69-4bd5-a6ae-0857c0d4eb64 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:31.920992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd694998-bd69-4bd5-a6ae-0857c0d4eb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:31.921338 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd694998-bd69-4bd5-a6ae-0857c0d4eb64 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:31.922205 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd694998-bd69-4bd5-a6ae-0857c0d4eb64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:31.924898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd694998-bd69-4bd5-a6ae-0857c0d4eb64 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:31.952450 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5b5bf-deec-4ae6-b0a0-cae4bd1620a8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:31.993758 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1216/2427] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:31 2022] POST /identity/v3/users => generated 307 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:31.994902 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd694998-bd69-4bd5-a6ae-0857c0d4eb64 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:32.015143 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd694998-bd69-4bd5-a6ae-0857c0d4eb64 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:32.023148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21472253-76cd-4681-af4f-e31151fed4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:32.024805 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21472253-76cd-4681-af4f-e31151fed4ba admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:32.025371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21472253-76cd-4681-af4f-e31151fed4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:32.025720 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21472253-76cd-4681-af4f-e31151fed4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:32.026939 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21472253-76cd-4681-af4f-e31151fed4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:32.028992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21472253-76cd-4681-af4f-e31151fed4ba admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:32.052629 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1217/2428] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:32.056891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21472253-76cd-4681-af4f-e31151fed4ba admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:32.068879 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21472253-76cd-4681-af4f-e31151fed4ba admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:32.071516 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5b5bf-deec-4ae6-b0a0-cae4bd1620a8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:32.074824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d1d95-3fcc-4e03-99ca-0de51d9a8037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:32.075676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d1d95-3fcc-4e03-99ca-0de51d9a8037 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:32.076035 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d1d95-3fcc-4e03-99ca-0de51d9a8037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:32.076318 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a65f-1908-460f-9d07-e1b1d00219a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:32.076466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d1d95-3fcc-4e03-99ca-0de51d9a8037 admin admin] PATH_INFO: `/v3/projects/749706b022784ff1b9eb9554327894bd/users/8a14d1b45bfc4d9bacd58fea73d7fd31/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:32.077317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a65f-1908-460f-9d07-e1b1d00219a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:32.077842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a65f-1908-460f-9d07-e1b1d00219a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:32.078328 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a65f-1908-460f-9d07-e1b1d00219a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:32.080378 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7699a65f-1908-460f-9d07-e1b1d00219a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:32.095624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699a65f-1908-460f-9d07-e1b1d00219a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:32.097710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699a65f-1908-460f-9d07-e1b1d00219a2 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:32.109744 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d1d95-3fcc-4e03-99ca-0de51d9a8037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=749706b022784ff1b9eb9554327894bd, user_id=8a14d1b45bfc4d9bacd58fea73d7fd31, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:32.110644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1212/2429] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:32.112653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d1d95-3fcc-4e03-99ca-0de51d9a8037 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:32.141002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1218/2430] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:32 2022] PUT /identity/v3/projects/749706b022784ff1b9eb9554327894bd/users/8a14d1b45bfc4d9bacd58fea73d7fd31/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:32.145032 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a65f-1908-460f-9d07-e1b1d00219a2 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:32.157559 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a65f-1908-460f-9d07-e1b1d00219a2 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:32.236353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56502a5a-bda6-4059-b2a0-4b05763bc1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:32.237279 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56502a5a-bda6-4059-b2a0-4b05763bc1bd admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:32.237609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56502a5a-bda6-4059-b2a0-4b05763bc1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:32.237887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56502a5a-bda6-4059-b2a0-4b05763bc1bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:32.238616 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56502a5a-bda6-4059-b2a0-4b05763bc1bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:32.240909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56502a5a-bda6-4059-b2a0-4b05763bc1bd admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:32.254288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1213/2431] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:50:32.258297 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d1d95-3fcc-4e03-99ca-0de51d9a8037 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:32.268815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d1d95-3fcc-4e03-99ca-0de51d9a8037 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:32.273448 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d252319-39a8-4c33-afdc-3dd143549de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:32.274148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d252319-39a8-4c33-afdc-3dd143549de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:32.274430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d252319-39a8-4c33-afdc-3dd143549de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:32.274720 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d252319-39a8-4c33-afdc-3dd143549de5 admin admin] PATH_INFO: `/v3/projects/749706b022784ff1b9eb9554327894bd/users/8a14d1b45bfc4d9bacd58fea73d7fd31/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:32.276682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d252319-39a8-4c33-afdc-3dd143549de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=749706b022784ff1b9eb9554327894bd, user_id=8a14d1b45bfc4d9bacd58fea73d7fd31, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:32.279353 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d252319-39a8-4c33-afdc-3dd143549de5 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:32.291691 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1219/2432] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:32 2022] PUT /identity/v3/projects/749706b022784ff1b9eb9554327894bd/users/8a14d1b45bfc4d9bacd58fea73d7fd31/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:33.228532 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34174824-2383-4f94-9153-4110423523c9 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:33.229195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34174824-2383-4f94-9153-4110423523c9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:33.229195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34174824-2383-4f94-9153-4110423523c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:33.296059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:33.301404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-34174824-2383-4f94-9153-4110423523c9 None None] MFA Rules not processed for user `8a14d1b45bfc4d9bacd58fea73d7fd31`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:33.303093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-34174824-2383-4f94-9153-4110423523c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:33.510474 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1214/2433] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:33 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:33.871852 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d252319-39a8-4c33-afdc-3dd143549de5 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:33.873731 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2d252319-39a8-4c33-afdc-3dd143549de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:33.980238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:34.009802 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d252319-39a8-4c33-afdc-3dd143549de5 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:34.022799 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22a0c6-f356-4c82-a0ae-e6d5302a94db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:34.023673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22a0c6-f356-4c82-a0ae-e6d5302a94db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:34.024763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22a0c6-f356-4c82-a0ae-e6d5302a94db admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:34.025605 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22a0c6-f356-4c82-a0ae-e6d5302a94db admin admin] PATH_INFO: `/v3/users/54bbc7809d024326901ca92bb7c04564` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:34.028649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a22a0c6-f356-4c82-a0ae-e6d5302a94db admin admin] RBAC: Authorizing `identity:delete_user(user_id=54bbc7809d024326901ca92bb7c04564)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:34.031934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a22a0c6-f356-4c82-a0ae-e6d5302a94db admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:34.094796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9a22a0c6-f356-4c82-a0ae-e6d5302a94db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54bbc7809d024326901ca92bb7c04564'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:34.114764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9a22a0c6-f356-4c82-a0ae-e6d5302a94db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54bbc7809d024326901ca92bb7c04564'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:34.115811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9a22a0c6-f356-4c82-a0ae-e6d5302a94db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54bbc7809d024326901ca92bb7c04564'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:34.128920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9a22a0c6-f356-4c82-a0ae-e6d5302a94db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54bbc7809d024326901ca92bb7c04564'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:34.140160 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9a22a0c6-f356-4c82-a0ae-e6d5302a94db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54bbc7809d024326901ca92bb7c04564'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:34.164756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1220/2434] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:33 2022] DELETE /identity/v3/users/54bbc7809d024326901ca92bb7c04564 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:34.167982 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743f5f4-cf76-45de-bee2-7bbc955cb098 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:34.168729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743f5f4-cf76-45de-bee2-7bbc955cb098 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:34.169244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743f5f4-cf76-45de-bee2-7bbc955cb098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:34.206996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5743f5f4-cf76-45de-bee2-7bbc955cb098 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:34.210614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5743f5f4-cf76-45de-bee2-7bbc955cb098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:34.318206 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22a0c6-f356-4c82-a0ae-e6d5302a94db admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:34.319859 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9a22a0c6-f356-4c82-a0ae-e6d5302a94db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:34.389367 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1215/2435] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:34.403637 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743f5f4-cf76-45de-bee2-7bbc955cb098 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:34.405119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5743f5f4-cf76-45de-bee2-7bbc955cb098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:34.521864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22a0c6-f356-4c82-a0ae-e6d5302a94db admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:34.524423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743f5f4-cf76-45de-bee2-7bbc955cb098 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:34.528526 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945f90e-4e23-428b-a5c1-9f601e43676f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2c5536eddd346d692822a2f3befb48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:34.529247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945f90e-4e23-428b-a5c1-9f601e43676f service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:34.529640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945f90e-4e23-428b-a5c1-9f601e43676f service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:34.529991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945f90e-4e23-428b-a5c1-9f601e43676f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:34.532466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b945f90e-4e23-428b-a5c1-9f601e43676f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:34.534107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae435736-601c-46e7-a532-3c1d9ffb70e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:34.534865 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae435736-601c-46e7-a532-3c1d9ffb70e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:34.535555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae435736-601c-46e7-a532-3c1d9ffb70e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:34.538562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae435736-601c-46e7-a532-3c1d9ffb70e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:34.559896 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae435736-601c-46e7-a532-3c1d9ffb70e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:34.563852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae435736-601c-46e7-a532-3c1d9ffb70e0 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:34.576440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945f90e-4e23-428b-a5c1-9f601e43676f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:34.578246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945f90e-4e23-428b-a5c1-9f601e43676f service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:34.591267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1216/2436] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:50:34.607728 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1221/2437] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:35.549264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae435736-601c-46e7-a532-3c1d9ffb70e0 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:35.550859 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ae435736-601c-46e7-a532-3c1d9ffb70e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:35.573115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae435736-601c-46e7-a532-3c1d9ffb70e0 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:35.599375 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30769508-8cf1-4c2e-b102-410cb9eaa7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:35.600364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30769508-8cf1-4c2e-b102-410cb9eaa7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:35.600832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30769508-8cf1-4c2e-b102-410cb9eaa7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:35.601282 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30769508-8cf1-4c2e-b102-410cb9eaa7fa admin admin] PATH_INFO: `/v3/projects/530f840de11e4324a99e9b5161d2a3f8` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:35.603412 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30769508-8cf1-4c2e-b102-410cb9eaa7fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=530f840de11e4324a99e9b5161d2a3f8)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:35.606597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30769508-8cf1-4c2e-b102-410cb9eaa7fa admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:35.714106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:35.745920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-30769508-8cf1-4c2e-b102-410cb9eaa7fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '530f840de11e4324a99e9b5161d2a3f8', 'request_id': 'req-30769508-8cf1-4c2e-b102-410cb9eaa7fa'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:35.762017 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-30769508-8cf1-4c2e-b102-410cb9eaa7fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '530f840de11e4324a99e9b5161d2a3f8', 'request_id': 'req-30769508-8cf1-4c2e-b102-410cb9eaa7fa'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:35.774609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-30769508-8cf1-4c2e-b102-410cb9eaa7fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '530f840de11e4324a99e9b5161d2a3f8', 'request_id': 'req-30769508-8cf1-4c2e-b102-410cb9eaa7fa'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:35.830280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-30769508-8cf1-4c2e-b102-410cb9eaa7fa admin admin] The token cache is being invalidate because project 530f840de11e4324a99e9b5161d2a3f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:35.830855 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-30769508-8cf1-4c2e-b102-410cb9eaa7fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:35.832307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-30769508-8cf1-4c2e-b102-410cb9eaa7fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:35.836362 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1217/2438] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:35 2022] DELETE /identity/v3/projects/530f840de11e4324a99e9b5161d2a3f8 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:50:35.842710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c62f9-6af0-43bc-a4b6-898263b713da None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:35.843285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c62f9-6af0-43bc-a4b6-898263b713da None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:35.843666 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c62f9-6af0-43bc-a4b6-898263b713da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:35.887022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4b0c62f9-6af0-43bc-a4b6-898263b713da None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:35.888482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4b0c62f9-6af0-43bc-a4b6-898263b713da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:35.982455 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30769508-8cf1-4c2e-b102-410cb9eaa7fa admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:35.985893 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-30769508-8cf1-4c2e-b102-410cb9eaa7fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:36.018854 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:36.059992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1222/2439] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:36.070216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa185540-a408-42ac-9f7c-bfddec2184cb None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:36.070653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa185540-a408-42ac-9f7c-bfddec2184cb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:36.071062 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa185540-a408-42ac-9f7c-bfddec2184cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:36.096115 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-aa185540-a408-42ac-9f7c-bfddec2184cb None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:36.098041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-aa185540-a408-42ac-9f7c-bfddec2184cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:36.111778 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1223/2440] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:36.116975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa185540-a408-42ac-9f7c-bfddec2184cb None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:36.135332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa185540-a408-42ac-9f7c-bfddec2184cb None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:36.149795 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc528a3-dac4-4218-8259-d32060016849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:36.151418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc528a3-dac4-4218-8259-d32060016849 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:36.151805 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc528a3-dac4-4218-8259-d32060016849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:36.152194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc528a3-dac4-4218-8259-d32060016849 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:36.153336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc528a3-dac4-4218-8259-d32060016849 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:50:36.153336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc528a3-dac4-4218-8259-d32060016849 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:36.155302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc528a3-dac4-4218-8259-d32060016849 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:36.178026 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1224/2441] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:50:36.179650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc528a3-dac4-4218-8259-d32060016849 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:36.191895 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc528a3-dac4-4218-8259-d32060016849 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:36.205007 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad6292-bbad-419c-8b49-463ea70ee6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:36.206371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad6292-bbad-419c-8b49-463ea70ee6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:36.206730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad6292-bbad-419c-8b49-463ea70ee6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:36.207146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad6292-bbad-419c-8b49-463ea70ee6f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:36.207895 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad6292-bbad-419c-8b49-463ea70ee6f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:36.210301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad6292-bbad-419c-8b49-463ea70ee6f7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:36.221698 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30769508-8cf1-4c2e-b102-410cb9eaa7fa admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:36.240479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc9850-2371-42eb-aa90-58d1576d07cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:36.241559 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc9850-2371-42eb-aa90-58d1576d07cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:36.242076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc9850-2371-42eb-aa90-58d1576d07cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:36.242439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc9850-2371-42eb-aa90-58d1576d07cb admin admin] PATH_INFO: `/v3/users/6d87144bec4c437cbb24335ce7726f17` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:36.244493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc9850-2371-42eb-aa90-58d1576d07cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d87144bec4c437cbb24335ce7726f17)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:36.248453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc9850-2371-42eb-aa90-58d1576d07cb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:36.283248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1225/2442] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:36 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:36.284502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad6292-bbad-419c-8b49-463ea70ee6f7 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:36.382367 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fefc9850-2371-42eb-aa90-58d1576d07cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d87144bec4c437cbb24335ce7726f17'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:36.407043 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fefc9850-2371-42eb-aa90-58d1576d07cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d87144bec4c437cbb24335ce7726f17'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:36.422037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fefc9850-2371-42eb-aa90-58d1576d07cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d87144bec4c437cbb24335ce7726f17'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:36.425946 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fefc9850-2371-42eb-aa90-58d1576d07cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d87144bec4c437cbb24335ce7726f17'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:36.427021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fefc9850-2371-42eb-aa90-58d1576d07cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d87144bec4c437cbb24335ce7726f17'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:36.450018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad6292-bbad-419c-8b49-463ea70ee6f7 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:36.466536 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989a065-9441-4e9b-8520-39aea286acc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:36.468763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989a065-9441-4e9b-8520-39aea286acc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:36.469807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989a065-9441-4e9b-8520-39aea286acc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:36.470617 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989a065-9441-4e9b-8520-39aea286acc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:36.473393 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989a065-9441-4e9b-8520-39aea286acc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:36.480694 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989a065-9441-4e9b-8520-39aea286acc6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:36.487281 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1218/2443] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:35 2022] DELETE /identity/v3/users/6d87144bec4c437cbb24335ce7726f17 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:50:36.525389 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1226/2444] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:36 2022] POST /identity/v3/users => generated 291 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:36.531010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc9850-2371-42eb-aa90-58d1576d07cb admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:36.533054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fefc9850-2371-42eb-aa90-58d1576d07cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:36.677700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:36.772466 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc9850-2371-42eb-aa90-58d1576d07cb admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:36.814587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43238f7f-5925-4ceb-a041-f6a8f2e8ef05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:36.815444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43238f7f-5925-4ceb-a041-f6a8f2e8ef05 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:36.815735 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43238f7f-5925-4ceb-a041-f6a8f2e8ef05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:36.816018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43238f7f-5925-4ceb-a041-f6a8f2e8ef05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:36.816927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43238f7f-5925-4ceb-a041-f6a8f2e8ef05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:36.819260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43238f7f-5925-4ceb-a041-f6a8f2e8ef05 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:36.850221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1219/2445] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:36.854840 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989a065-9441-4e9b-8520-39aea286acc6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:36.869862 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43238f7f-5925-4ceb-a041-f6a8f2e8ef05 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:36.873360 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989a065-9441-4e9b-8520-39aea286acc6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:36.887028 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-43238f7f-5925-4ceb-a041-f6a8f2e8ef05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:36.898531 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a74de-292a-41be-b0e2-080857d56e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:36.901790 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a74de-292a-41be-b0e2-080857d56e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:36.902256 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a74de-292a-41be-b0e2-080857d56e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:36.902696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a74de-292a-41be-b0e2-080857d56e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:36.904554 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a74de-292a-41be-b0e2-080857d56e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:36.909041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a74de-292a-41be-b0e2-080857d56e88 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:36.920870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43238f7f-5925-4ceb-a041-f6a8f2e8ef05 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:36.933828 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33af4d-5fed-43d1-ba3d-90af268ed789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:36.936373 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33af4d-5fed-43d1-ba3d-90af268ed789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:36.936838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33af4d-5fed-43d1-ba3d-90af268ed789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:36.938729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33af4d-5fed-43d1-ba3d-90af268ed789 admin admin] PATH_INFO: `/v3/projects/a0163ced4c2f49c09b1a7e17814fb6c1` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:36.940891 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33af4d-5fed-43d1-ba3d-90af268ed789 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0163ced4c2f49c09b1a7e17814fb6c1)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:36.941360 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1227/2446] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:36.944118 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a74de-292a-41be-b0e2-080857d56e88 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:36.947998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33af4d-5fed-43d1-ba3d-90af268ed789 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:36.977486 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a74de-292a-41be-b0e2-080857d56e88 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:36.991539 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c673158-29b6-426e-abf2-e6014ec1f248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:36.994344 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c673158-29b6-426e-abf2-e6014ec1f248 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:36.994754 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c673158-29b6-426e-abf2-e6014ec1f248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:36.995144 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c673158-29b6-426e-abf2-e6014ec1f248 admin admin] PATH_INFO: `/v3/projects/ac7eca525bbe4443a7b99e0aa921050e/users/538d552b04a8446094d2e84952804795/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:37.037901 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c673158-29b6-426e-abf2-e6014ec1f248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac7eca525bbe4443a7b99e0aa921050e, user_id=538d552b04a8446094d2e84952804795, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:37.043244 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c673158-29b6-426e-abf2-e6014ec1f248 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:37.070684 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1228/2447] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:36 2022] PUT /identity/v3/projects/ac7eca525bbe4443a7b99e0aa921050e/users/538d552b04a8446094d2e84952804795/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:37.075535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b136f41-8b47-4327-aee1-0ca0f62cca8d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:37.076861 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b136f41-8b47-4327-aee1-0ca0f62cca8d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:37.077361 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b136f41-8b47-4327-aee1-0ca0f62cca8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:37.079444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5e33af4d-5fed-43d1-ba3d-90af268ed789 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0163ced4c2f49c09b1a7e17814fb6c1', 'request_id': 'req-5e33af4d-5fed-43d1-ba3d-90af268ed789'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:37.095860 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5e33af4d-5fed-43d1-ba3d-90af268ed789 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0163ced4c2f49c09b1a7e17814fb6c1', 'request_id': 'req-5e33af4d-5fed-43d1-ba3d-90af268ed789'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:37.107423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0b136f41-8b47-4327-aee1-0ca0f62cca8d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:37.108882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0b136f41-8b47-4327-aee1-0ca0f62cca8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:37.114094 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5e33af4d-5fed-43d1-ba3d-90af268ed789 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0163ced4c2f49c09b1a7e17814fb6c1', 'request_id': 'req-5e33af4d-5fed-43d1-ba3d-90af268ed789'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:37.140882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5e33af4d-5fed-43d1-ba3d-90af268ed789 admin admin] The token cache is being invalidate because project a0163ced4c2f49c09b1a7e17814fb6c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:37.141498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5e33af4d-5fed-43d1-ba3d-90af268ed789 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:37.144658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5e33af4d-5fed-43d1-ba3d-90af268ed789 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:37.148975 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1220/2448] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:36 2022] DELETE /identity/v3/projects/a0163ced4c2f49c09b1a7e17814fb6c1 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:37.153359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33af4d-5fed-43d1-ba3d-90af268ed789 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:37.156444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5e33af4d-5fed-43d1-ba3d-90af268ed789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:37.247584 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1229/2449] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:37.259843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b136f41-8b47-4327-aee1-0ca0f62cca8d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:37.261316 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0b136f41-8b47-4327-aee1-0ca0f62cca8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:37.271403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:37.354339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33af4d-5fed-43d1-ba3d-90af268ed789 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:37.397121 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e381078-a5c4-43c5-93cc-1e267feaaf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:37.398357 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e381078-a5c4-43c5-93cc-1e267feaaf39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:37.398839 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e381078-a5c4-43c5-93cc-1e267feaaf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:37.399348 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e381078-a5c4-43c5-93cc-1e267feaaf39 admin admin] PATH_INFO: `/v3/users/8a14d1b45bfc4d9bacd58fea73d7fd31` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:37.402171 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e381078-a5c4-43c5-93cc-1e267feaaf39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a14d1b45bfc4d9bacd58fea73d7fd31)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:37.412608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e381078-a5c4-43c5-93cc-1e267feaaf39 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:37.422764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b136f41-8b47-4327-aee1-0ca0f62cca8d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:37.432633 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8333930f-24cb-4371-ab0e-7a6205a1e485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:37.434611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333930f-24cb-4371-ab0e-7a6205a1e485 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:37.435626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333930f-24cb-4371-ab0e-7a6205a1e485 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:37.437060 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333930f-24cb-4371-ab0e-7a6205a1e485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:37.456340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8333930f-24cb-4371-ab0e-7a6205a1e485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:37.459155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8333930f-24cb-4371-ab0e-7a6205a1e485 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:37.484907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1230/2450] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:37.525172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9e381078-a5c4-43c5-93cc-1e267feaaf39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a14d1b45bfc4d9bacd58fea73d7fd31'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:37.535803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9e381078-a5c4-43c5-93cc-1e267feaaf39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a14d1b45bfc4d9bacd58fea73d7fd31'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:37.552690 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9e381078-a5c4-43c5-93cc-1e267feaaf39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a14d1b45bfc4d9bacd58fea73d7fd31'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:37.555516 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9e381078-a5c4-43c5-93cc-1e267feaaf39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a14d1b45bfc4d9bacd58fea73d7fd31'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:37.556608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9e381078-a5c4-43c5-93cc-1e267feaaf39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a14d1b45bfc4d9bacd58fea73d7fd31'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:37.563913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:37.606156 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1221/2451] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:37 2022] DELETE /identity/v3/users/8a14d1b45bfc4d9bacd58fea73d7fd31 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:38.645963 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8333930f-24cb-4371-ab0e-7a6205a1e485 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:38.658560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8333930f-24cb-4371-ab0e-7a6205a1e485 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:38.815517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224b193-48e2-4da7-a770-2bf079bb0732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:38.816355 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224b193-48e2-4da7-a770-2bf079bb0732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:38.817070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224b193-48e2-4da7-a770-2bf079bb0732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:38.817070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224b193-48e2-4da7-a770-2bf079bb0732 admin admin] PATH_INFO: `/v3/projects/749706b022784ff1b9eb9554327894bd` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:38.818972 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224b193-48e2-4da7-a770-2bf079bb0732 admin admin] RBAC: Authorizing `identity:delete_project(project_id=749706b022784ff1b9eb9554327894bd)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:38.821384 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224b193-48e2-4da7-a770-2bf079bb0732 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:38.832119 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e381078-a5c4-43c5-93cc-1e267feaaf39 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:38.834247 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9e381078-a5c4-43c5-93cc-1e267feaaf39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:38.865665 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e381078-a5c4-43c5-93cc-1e267feaaf39 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:38.873116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c24ee-0c77-45bf-90ef-986c73d12933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:38.874207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c24ee-0c77-45bf-90ef-986c73d12933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:38.874546 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c24ee-0c77-45bf-90ef-986c73d12933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:38.875004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c24ee-0c77-45bf-90ef-986c73d12933 admin admin] PATH_INFO: `/v3/users/956aff831fa04d29ba5cffb88e73f4d2` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:38.876651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c24ee-0c77-45bf-90ef-986c73d12933 admin admin] RBAC: Authorizing `identity:delete_user(user_id=956aff831fa04d29ba5cffb88e73f4d2)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:38.881031 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c24ee-0c77-45bf-90ef-986c73d12933 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:38.902105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2224b193-48e2-4da7-a770-2bf079bb0732 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '749706b022784ff1b9eb9554327894bd', 'request_id': 'req-2224b193-48e2-4da7-a770-2bf079bb0732'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:38.911982 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2224b193-48e2-4da7-a770-2bf079bb0732 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '749706b022784ff1b9eb9554327894bd', 'request_id': 'req-2224b193-48e2-4da7-a770-2bf079bb0732'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:38.920286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2224b193-48e2-4da7-a770-2bf079bb0732 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '749706b022784ff1b9eb9554327894bd', 'request_id': 'req-2224b193-48e2-4da7-a770-2bf079bb0732'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:38.934211 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2224b193-48e2-4da7-a770-2bf079bb0732 admin admin] The token cache is being invalidate because project 749706b022784ff1b9eb9554327894bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:38.934505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2224b193-48e2-4da7-a770-2bf079bb0732 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:38.935193 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2224b193-48e2-4da7-a770-2bf079bb0732 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:38.937243 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1231/2452] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:38 2022] DELETE /identity/v3/projects/749706b022784ff1b9eb9554327894bd => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:38.939776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-eb1c24ee-0c77-45bf-90ef-986c73d12933 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '956aff831fa04d29ba5cffb88e73f4d2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:38.941703 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b251480-004d-496f-9fd5-9f02aa75c226 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:38.942105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b251480-004d-496f-9fd5-9f02aa75c226 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:38.942505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b251480-004d-496f-9fd5-9f02aa75c226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:38.949148 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-eb1c24ee-0c77-45bf-90ef-986c73d12933 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '956aff831fa04d29ba5cffb88e73f4d2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:38.958573 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-eb1c24ee-0c77-45bf-90ef-986c73d12933 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '956aff831fa04d29ba5cffb88e73f4d2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:38.959471 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-eb1c24ee-0c77-45bf-90ef-986c73d12933 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '956aff831fa04d29ba5cffb88e73f4d2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:38.960148 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-eb1c24ee-0c77-45bf-90ef-986c73d12933 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '956aff831fa04d29ba5cffb88e73f4d2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:38.965300 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6b251480-004d-496f-9fd5-9f02aa75c226 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:38.966716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6b251480-004d-496f-9fd5-9f02aa75c226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:38.983929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1222/2453] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:38 2022] DELETE /identity/v3/users/956aff831fa04d29ba5cffb88e73f4d2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:39.010751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:39.063003 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1232/2454] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:39.068972 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4c13a-7882-4ff9-a20b-7ac10572c446 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:39.069281 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4c13a-7882-4ff9-a20b-7ac10572c446 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:39.069586 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4c13a-7882-4ff9-a20b-7ac10572c446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:39.096199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a2f4c13a-7882-4ff9-a20b-7ac10572c446 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:39.097738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a2f4c13a-7882-4ff9-a20b-7ac10572c446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:39.171715 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1223/2455] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:39.173054 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b251480-004d-496f-9fd5-9f02aa75c226 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:39.184962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b251480-004d-496f-9fd5-9f02aa75c226 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:39.190156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158557f-94e0-40d3-aa44-b8a5f6315db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:39.191462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158557f-94e0-40d3-aa44-b8a5f6315db8 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:39.191462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158557f-94e0-40d3-aa44-b8a5f6315db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:39.191714 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158557f-94e0-40d3-aa44-b8a5f6315db8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:39.192635 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158557f-94e0-40d3-aa44-b8a5f6315db8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:50:39.193009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158557f-94e0-40d3-aa44-b8a5f6315db8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:39.195622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158557f-94e0-40d3-aa44-b8a5f6315db8 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:39.213256 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1233/2456] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:50:39.214875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4c13a-7882-4ff9-a20b-7ac10572c446 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:39.225629 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4c13a-7882-4ff9-a20b-7ac10572c446 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:39.234842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ec019-c94d-4a05-b0b9-83b60327fcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:39.235821 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ec019-c94d-4a05-b0b9-83b60327fcde admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:39.236264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ec019-c94d-4a05-b0b9-83b60327fcde admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:39.236748 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ec019-c94d-4a05-b0b9-83b60327fcde admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:39.237642 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ec019-c94d-4a05-b0b9-83b60327fcde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:39.241009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ec019-c94d-4a05-b0b9-83b60327fcde admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:39.281644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1224/2457] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:39 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:39.285544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158557f-94e0-40d3-aa44-b8a5f6315db8 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:39.296929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158557f-94e0-40d3-aa44-b8a5f6315db8 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:39.369759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ffcb7-5099-4d1e-b1f8-72b0dff03709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:39.370478 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ffcb7-5099-4d1e-b1f8-72b0dff03709 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:39.370869 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ffcb7-5099-4d1e-b1f8-72b0dff03709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:39.371248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ffcb7-5099-4d1e-b1f8-72b0dff03709 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:39.372026 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ffcb7-5099-4d1e-b1f8-72b0dff03709 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:39.374320 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ffcb7-5099-4d1e-b1f8-72b0dff03709 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:39.399468 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1234/2458] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:39 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:39.404439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ec019-c94d-4a05-b0b9-83b60327fcde admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:39.422693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ec019-c94d-4a05-b0b9-83b60327fcde admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:39.431151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fdc3f-1d5c-4719-b08f-c11b275cfc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:39.433269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fdc3f-1d5c-4719-b08f-c11b275cfc7b admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:39.433764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fdc3f-1d5c-4719-b08f-c11b275cfc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:39.434739 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fdc3f-1d5c-4719-b08f-c11b275cfc7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:39.435647 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0fdc3f-1d5c-4719-b08f-c11b275cfc7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:39.438356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0fdc3f-1d5c-4719-b08f-c11b275cfc7b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:39.451561 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1225/2459] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:39.455820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ffcb7-5099-4d1e-b1f8-72b0dff03709 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:39.468986 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ffcb7-5099-4d1e-b1f8-72b0dff03709 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:39.478953 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fbaff-e259-4564-81aa-e93af1c2bec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:39.479616 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fbaff-e259-4564-81aa-e93af1c2bec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:39.480126 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fbaff-e259-4564-81aa-e93af1c2bec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:39.480486 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fbaff-e259-4564-81aa-e93af1c2bec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:39.481278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fbaff-e259-4564-81aa-e93af1c2bec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:39.483395 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fbaff-e259-4564-81aa-e93af1c2bec6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:39.500135 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1235/2460] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:50:39.501495 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fdc3f-1d5c-4719-b08f-c11b275cfc7b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:39.513404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fdc3f-1d5c-4719-b08f-c11b275cfc7b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:39.518520 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e515c-cf52-4ffa-b373-6c2d753749a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:39.519312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e515c-cf52-4ffa-b373-6c2d753749a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:39.519798 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e515c-cf52-4ffa-b373-6c2d753749a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:39.520147 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e515c-cf52-4ffa-b373-6c2d753749a6 admin admin] PATH_INFO: `/v3/projects/68c5e7a820a04dcd9230b68fed62eac9/users/fb599a8d773e4bb2802b41c64dc5258e/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:39.537361 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e515c-cf52-4ffa-b373-6c2d753749a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68c5e7a820a04dcd9230b68fed62eac9, user_id=fb599a8d773e4bb2802b41c64dc5258e, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:39.540390 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e515c-cf52-4ffa-b373-6c2d753749a6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:39.551255 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:39.561061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1226/2461] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:39 2022] PUT /identity/v3/projects/68c5e7a820a04dcd9230b68fed62eac9/users/fb599a8d773e4bb2802b41c64dc5258e/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:50:39.573308 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd8cd-e340-47bd-8201-e4b855edc466 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:39.573651 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd8cd-e340-47bd-8201-e4b855edc466 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:39.574115 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd8cd-e340-47bd-8201-e4b855edc466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:39.608321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c29bd8cd-e340-47bd-8201-e4b855edc466 None None] MFA Rules not processed for user `fb599a8d773e4bb2802b41c64dc5258e`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:39.610122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c29bd8cd-e340-47bd-8201-e4b855edc466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:39.708874 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1236/2462] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:39.713704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e515c-cf52-4ffa-b373-6c2d753749a6 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:39.716953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c89e515c-cf52-4ffa-b373-6c2d753749a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:39.719180 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bd8cd-e340-47bd-8201-e4b855edc466 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:39.783730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:39.862040 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e515c-cf52-4ffa-b373-6c2d753749a6 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:39.867331 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bd8cd-e340-47bd-8201-e4b855edc466 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:39.876371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e420c-ee1f-4ecb-b2db-aca483000b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:39.876587 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dc436-9efc-4f2f-99c2-b0a89e5d3884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:39.877014 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e420c-ee1f-4ecb-b2db-aca483000b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:39.877139 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dc436-9efc-4f2f-99c2-b0a89e5d3884 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:39.877330 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e420c-ee1f-4ecb-b2db-aca483000b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:39.877438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dc436-9efc-4f2f-99c2-b0a89e5d3884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:39.877652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dc436-9efc-4f2f-99c2-b0a89e5d3884 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:39.877753 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e420c-ee1f-4ecb-b2db-aca483000b22 admin admin] PATH_INFO: `/v3/projects/40540f8322a54133bc32e1af000810ad` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:39.878404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52dc436-9efc-4f2f-99c2-b0a89e5d3884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:39.880341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52dc436-9efc-4f2f-99c2-b0a89e5d3884 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:39.880877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e420c-ee1f-4ecb-b2db-aca483000b22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40540f8322a54133bc32e1af000810ad)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:39.882838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e420c-ee1f-4ecb-b2db-aca483000b22 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:39.927591 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1237/2463] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:39 2022] POST /identity/v3/projects => generated 373 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:39.929248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dc436-9efc-4f2f-99c2-b0a89e5d3884 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:39.965735 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-311e420c-ee1f-4ecb-b2db-aca483000b22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40540f8322a54133bc32e1af000810ad', 'request_id': 'req-311e420c-ee1f-4ecb-b2db-aca483000b22'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:39.974148 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-311e420c-ee1f-4ecb-b2db-aca483000b22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40540f8322a54133bc32e1af000810ad', 'request_id': 'req-311e420c-ee1f-4ecb-b2db-aca483000b22'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:39.995287 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-311e420c-ee1f-4ecb-b2db-aca483000b22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40540f8322a54133bc32e1af000810ad', 'request_id': 'req-311e420c-ee1f-4ecb-b2db-aca483000b22'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:40.018258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-311e420c-ee1f-4ecb-b2db-aca483000b22 admin admin] The token cache is being invalidate because project 40540f8322a54133bc32e1af000810ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:40.018551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-311e420c-ee1f-4ecb-b2db-aca483000b22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:40.019730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-311e420c-ee1f-4ecb-b2db-aca483000b22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:40.022866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1227/2464] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:39 2022] DELETE /identity/v3/projects/40540f8322a54133bc32e1af000810ad => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:40.031115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6232794-17ae-4971-9147-ab4fad876fee None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.031115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6232794-17ae-4971-9147-ab4fad876fee None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.031480 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6232794-17ae-4971-9147-ab4fad876fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.051041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dc436-9efc-4f2f-99c2-b0a89e5d3884 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:40.056727 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e6232794-17ae-4971-9147-ab4fad876fee None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:40.058196 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e6232794-17ae-4971-9147-ab4fad876fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:40.169236 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23602d9a-4cbb-499c-bbb8-a558b6625630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:40.170913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23602d9a-4cbb-499c-bbb8-a558b6625630 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.171373 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23602d9a-4cbb-499c-bbb8-a558b6625630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.171817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23602d9a-4cbb-499c-bbb8-a558b6625630 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.172497 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23602d9a-4cbb-499c-bbb8-a558b6625630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:40.173116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1228/2465] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:40.175290 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23602d9a-4cbb-499c-bbb8-a558b6625630 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:40.181740 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd0eb5-0cd7-4ecc-b7e6-b4974189585f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.182022 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd0eb5-0cd7-4ecc-b7e6-b4974189585f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.182385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd0eb5-0cd7-4ecc-b7e6-b4974189585f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.210464 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1238/2466] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:39 2022] POST /identity/v3/users => generated 293 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:40.214132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23602d9a-4cbb-499c-bbb8-a558b6625630 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:40.216008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-23602d9a-4cbb-499c-bbb8-a558b6625630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:40.221890 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8dbd0eb5-0cd7-4ecc-b7e6-b4974189585f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:40.223480 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8dbd0eb5-0cd7-4ecc-b7e6-b4974189585f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:40.234616 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23602d9a-4cbb-499c-bbb8-a558b6625630 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:40.235285 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1229/2467] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:40.240079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd0eb5-0cd7-4ecc-b7e6-b4974189585f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:40.240632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf406e-678f-42f5-9520-e69dfa60e945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:40.241346 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf406e-678f-42f5-9520-e69dfa60e945 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.241740 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf406e-678f-42f5-9520-e69dfa60e945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.242083 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf406e-678f-42f5-9520-e69dfa60e945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.243475 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf406e-678f-42f5-9520-e69dfa60e945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:40.245322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf406e-678f-42f5-9520-e69dfa60e945 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:40.250842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd0eb5-0cd7-4ecc-b7e6-b4974189585f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:40.257040 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e2baf-0948-4645-8a42-daa68ec831e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:40.257718 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e2baf-0948-4645-8a42-daa68ec831e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.258000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e2baf-0948-4645-8a42-daa68ec831e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.259557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e2baf-0948-4645-8a42-daa68ec831e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.260280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e2baf-0948-4645-8a42-daa68ec831e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:50:40.261083 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e2baf-0948-4645-8a42-daa68ec831e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:40.262935 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1239/2468] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:40.263527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e2baf-0948-4645-8a42-daa68ec831e5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:40.267193 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf406e-678f-42f5-9520-e69dfa60e945 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:40.276324 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1230/2469] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:50:40.283477 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e2baf-0948-4645-8a42-daa68ec831e5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:40.287311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf406e-678f-42f5-9520-e69dfa60e945 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:40.293389 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8ae7f-839d-403d-9aa1-01a85dbe5f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:40.294654 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8ae7f-839d-403d-9aa1-01a85dbe5f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.294947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8ae7f-839d-403d-9aa1-01a85dbe5f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.295227 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8ae7f-839d-403d-9aa1-01a85dbe5f6f admin admin] PATH_INFO: `/v3/projects/f1251e7ec64b44378ec3f90d7d60b31a/users/4a7cd4321e5746e1a9166794e293a5ac/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.298494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e2baf-0948-4645-8a42-daa68ec831e5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:40.304388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a01b4-05ff-47f4-b040-306c29bff497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:40.305098 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a01b4-05ff-47f4-b040-306c29bff497 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.306001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a01b4-05ff-47f4-b040-306c29bff497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.306352 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a01b4-05ff-47f4-b040-306c29bff497 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.307096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a01b4-05ff-47f4-b040-306c29bff497 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:40.309719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a01b4-05ff-47f4-b040-306c29bff497 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:40.316284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:40.319784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8ae7f-839d-403d-9aa1-01a85dbe5f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1251e7ec64b44378ec3f90d7d60b31a, user_id=4a7cd4321e5746e1a9166794e293a5ac, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:40.323993 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8ae7f-839d-403d-9aa1-01a85dbe5f6f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:40.342007 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1240/2470] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:40 2022] PUT /identity/v3/projects/f1251e7ec64b44378ec3f90d7d60b31a/users/4a7cd4321e5746e1a9166794e293a5ac/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:50:40.346085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1231/2471] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:40 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:40.347080 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8ae7f-839d-403d-9aa1-01a85dbe5f6f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:40.350195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a01b4-05ff-47f4-b040-306c29bff497 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:40.370469 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:40.426085 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8ae7f-839d-403d-9aa1-01a85dbe5f6f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:40.442960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a01b4-05ff-47f4-b040-306c29bff497 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:40.448190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7da99-dcee-4e9b-af50-9ce68393a863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:40.448849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7da99-dcee-4e9b-af50-9ce68393a863 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.449226 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7da99-dcee-4e9b-af50-9ce68393a863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.449653 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7da99-dcee-4e9b-af50-9ce68393a863 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.450522 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7da99-dcee-4e9b-af50-9ce68393a863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:40.452745 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7da99-dcee-4e9b-af50-9ce68393a863 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:40.485761 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1232/2472] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:40 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:40.487339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7da99-dcee-4e9b-af50-9ce68393a863 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:40.509340 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7da99-dcee-4e9b-af50-9ce68393a863 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:40.512122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61612df4-2abb-4778-a59f-c950a07e2c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:40.512839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61612df4-2abb-4778-a59f-c950a07e2c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.513741 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61612df4-2abb-4778-a59f-c950a07e2c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.514208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61612df4-2abb-4778-a59f-c950a07e2c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.514989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61612df4-2abb-4778-a59f-c950a07e2c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:40.516733 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61612df4-2abb-4778-a59f-c950a07e2c40 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:40.516859 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a3816-8869-413e-91c0-ec6b8b72bf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:40.517553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a3816-8869-413e-91c0-ec6b8b72bf19 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.517810 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a3816-8869-413e-91c0-ec6b8b72bf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.518082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a3816-8869-413e-91c0-ec6b8b72bf19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.519208 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a3816-8869-413e-91c0-ec6b8b72bf19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:40.521312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a3816-8869-413e-91c0-ec6b8b72bf19 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:40.528798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1241/2473] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:40.532519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61612df4-2abb-4778-a59f-c950a07e2c40 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:40.540800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1233/2474] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:40.541926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a3816-8869-413e-91c0-ec6b8b72bf19 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:40.546410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61612df4-2abb-4778-a59f-c950a07e2c40 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:40.553144 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5fc72-d5e5-48c4-93af-6c24bbd83a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:40.553973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a3816-8869-413e-91c0-ec6b8b72bf19 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:40.554555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5fc72-d5e5-48c4-93af-6c24bbd83a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.555112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5fc72-d5e5-48c4-93af-6c24bbd83a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.556307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5fc72-d5e5-48c4-93af-6c24bbd83a2b admin admin] PATH_INFO: `/v3/projects/f1251e7ec64b44378ec3f90d7d60b31a/users/4a7cd4321e5746e1a9166794e293a5ac/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.559952 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5fc72-d5e5-48c4-93af-6c24bbd83a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1251e7ec64b44378ec3f90d7d60b31a, user_id=4a7cd4321e5746e1a9166794e293a5ac, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:40.561378 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e565843-5342-45f6-906b-0b97a7fa6ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:40.562134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e565843-5342-45f6-906b-0b97a7fa6ecd admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.563027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e565843-5342-45f6-906b-0b97a7fa6ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.563390 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e565843-5342-45f6-906b-0b97a7fa6ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.563693 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5fc72-d5e5-48c4-93af-6c24bbd83a2b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:40.564107 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e565843-5342-45f6-906b-0b97a7fa6ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:40.566260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e565843-5342-45f6-906b-0b97a7fa6ecd admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:40.576521 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1242/2475] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:40 2022] PUT /identity/v3/projects/f1251e7ec64b44378ec3f90d7d60b31a/users/4a7cd4321e5746e1a9166794e293a5ac/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:50:40.587713 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996a886-19b4-4bab-94f9-62d6b10bcf33 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.588258 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996a886-19b4-4bab-94f9-62d6b10bcf33 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.588784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996a886-19b4-4bab-94f9-62d6b10bcf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.590875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1234/2476] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:40.594780 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e565843-5342-45f6-906b-0b97a7fa6ecd admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:40.608537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e565843-5342-45f6-906b-0b97a7fa6ecd admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:40.629984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9996a886-19b4-4bab-94f9-62d6b10bcf33 None None] MFA Rules not processed for user `4a7cd4321e5746e1a9166794e293a5ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:40.631686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9996a886-19b4-4bab-94f9-62d6b10bcf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:40.704010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9092e9-07ae-44a3-9704-a53e9adb3c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:40.705112 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9092e9-07ae-44a3-9704-a53e9adb3c11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.706244 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9092e9-07ae-44a3-9704-a53e9adb3c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.706724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9092e9-07ae-44a3-9704-a53e9adb3c11 admin admin] PATH_INFO: `/v3/projects/9559a93e928e48108b818ac3bb239e48/users/88cfa5e4719a4eeb94ac3c2175f680ed/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.730908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9092e9-07ae-44a3-9704-a53e9adb3c11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9559a93e928e48108b818ac3bb239e48, user_id=88cfa5e4719a4eeb94ac3c2175f680ed, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:40.735870 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9092e9-07ae-44a3-9704-a53e9adb3c11 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:40.757901 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1243/2477] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:40.769314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996a886-19b4-4bab-94f9-62d6b10bcf33 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:40.771629 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9996a886-19b4-4bab-94f9-62d6b10bcf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:40.779466 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1235/2478] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:40 2022] PUT /identity/v3/projects/9559a93e928e48108b818ac3bb239e48/users/88cfa5e4719a4eeb94ac3c2175f680ed/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:40.791283 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695a943-4c15-4662-84f4-542de168eedf None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.791630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695a943-4c15-4662-84f4-542de168eedf None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.791866 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695a943-4c15-4662-84f4-542de168eedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.831049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4695a943-4c15-4662-84f4-542de168eedf None None] MFA Rules not processed for user `88cfa5e4719a4eeb94ac3c2175f680ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:40.839183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4695a943-4c15-4662-84f4-542de168eedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:40.853717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:40.899796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996a886-19b4-4bab-94f9-62d6b10bcf33 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:40.909151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:40.985325 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68aca4-f2f6-4745-9e53-7532cc5bd182 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:40.986158 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68aca4-f2f6-4745-9e53-7532cc5bd182 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:40.986806 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68aca4-f2f6-4745-9e53-7532cc5bd182 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:40.988798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68aca4-f2f6-4745-9e53-7532cc5bd182 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:40.997453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1236/2479] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:41.009354 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695a943-4c15-4662-84f4-542de168eedf None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:41.010937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4695a943-4c15-4662-84f4-542de168eedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:41.112730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68aca4-f2f6-4745-9e53-7532cc5bd182 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:41.114992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68aca4-f2f6-4745-9e53-7532cc5bd182 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:41.127444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1244/2480] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:41.147044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695a943-4c15-4662-84f4-542de168eedf None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:41.154062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedda37b-94a3-4ba0-bc66-4626fcf185be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:41.155863 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedda37b-94a3-4ba0-bc66-4626fcf185be service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:41.156200 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedda37b-94a3-4ba0-bc66-4626fcf185be service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:41.156418 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedda37b-94a3-4ba0-bc66-4626fcf185be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:41.168162 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedda37b-94a3-4ba0-bc66-4626fcf185be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:41.171272 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedda37b-94a3-4ba0-bc66-4626fcf185be service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:41.225876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1237/2481] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:50:41.871572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68aca4-f2f6-4745-9e53-7532cc5bd182 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:42.048182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68aca4-f2f6-4745-9e53-7532cc5bd182 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:42.069712 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec3e32-7acd-4e3e-99c0-f302404a287d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:42.070410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec3e32-7acd-4e3e-99c0-f302404a287d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:42.070694 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec3e32-7acd-4e3e-99c0-f302404a287d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:42.072975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec3e32-7acd-4e3e-99c0-f302404a287d admin admin] PATH_INFO: `/v3/users/fb599a8d773e4bb2802b41c64dc5258e` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:42.074700 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec3e32-7acd-4e3e-99c0-f302404a287d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb599a8d773e4bb2802b41c64dc5258e)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:42.076756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec3e32-7acd-4e3e-99c0-f302404a287d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:42.148102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:42.159175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3fec3e32-7acd-4e3e-99c0-f302404a287d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb599a8d773e4bb2802b41c64dc5258e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:42.177677 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3fec3e32-7acd-4e3e-99c0-f302404a287d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb599a8d773e4bb2802b41c64dc5258e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:42.178723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3fec3e32-7acd-4e3e-99c0-f302404a287d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb599a8d773e4bb2802b41c64dc5258e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:42.189781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3fec3e32-7acd-4e3e-99c0-f302404a287d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb599a8d773e4bb2802b41c64dc5258e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:42.201412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3fec3e32-7acd-4e3e-99c0-f302404a287d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb599a8d773e4bb2802b41c64dc5258e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:42.215403 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb1ce2-129a-4409-9d5a-17b547944eb0 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:42.216149 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb1ce2-129a-4409-9d5a-17b547944eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:42.216149 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb1ce2-129a-4409-9d5a-17b547944eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:42.219426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1245/2482] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:41 2022] DELETE /identity/v3/users/fb599a8d773e4bb2802b41c64dc5258e => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:42.248205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e3fb1ce2-129a-4409-9d5a-17b547944eb0 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:42.250141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e3fb1ce2-129a-4409-9d5a-17b547944eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:42.339937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1238/2483] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:42.351929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec3e32-7acd-4e3e-99c0-f302404a287d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:42.355162 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3fec3e32-7acd-4e3e-99c0-f302404a287d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:42.427953 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec3e32-7acd-4e3e-99c0-f302404a287d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:42.433794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f9620-fbe5-44ac-8824-6733a54f41b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:42.435204 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f9620-fbe5-44ac-8824-6733a54f41b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:42.435567 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f9620-fbe5-44ac-8824-6733a54f41b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:42.436062 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f9620-fbe5-44ac-8824-6733a54f41b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:42.447909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f9620-fbe5-44ac-8824-6733a54f41b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:42.449478 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f9620-fbe5-44ac-8824-6733a54f41b0 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:42.464412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1246/2484] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:50:42.846757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb1ce2-129a-4409-9d5a-17b547944eb0 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:42.848629 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e3fb1ce2-129a-4409-9d5a-17b547944eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:42.873378 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb1ce2-129a-4409-9d5a-17b547944eb0 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:42.888130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ac260-71a2-48b9-b280-1ffebf35b408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:42.889038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ac260-71a2-48b9-b280-1ffebf35b408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:42.889418 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ac260-71a2-48b9-b280-1ffebf35b408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:42.889907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ac260-71a2-48b9-b280-1ffebf35b408 admin admin] PATH_INFO: `/v3/projects/68c5e7a820a04dcd9230b68fed62eac9` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:42.891299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ac260-71a2-48b9-b280-1ffebf35b408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68c5e7a820a04dcd9230b68fed62eac9)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:42.893581 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ac260-71a2-48b9-b280-1ffebf35b408 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:42.977990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:42.983878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b22ac260-71a2-48b9-b280-1ffebf35b408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68c5e7a820a04dcd9230b68fed62eac9', 'request_id': 'req-b22ac260-71a2-48b9-b280-1ffebf35b408'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:42.999982 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b22ac260-71a2-48b9-b280-1ffebf35b408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68c5e7a820a04dcd9230b68fed62eac9', 'request_id': 'req-b22ac260-71a2-48b9-b280-1ffebf35b408'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:43.015311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b22ac260-71a2-48b9-b280-1ffebf35b408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68c5e7a820a04dcd9230b68fed62eac9', 'request_id': 'req-b22ac260-71a2-48b9-b280-1ffebf35b408'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:43.031430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b22ac260-71a2-48b9-b280-1ffebf35b408 admin admin] The token cache is being invalidate because project 68c5e7a820a04dcd9230b68fed62eac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:43.032596 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b22ac260-71a2-48b9-b280-1ffebf35b408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:43.035062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b22ac260-71a2-48b9-b280-1ffebf35b408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:43.041296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f9620-fbe5-44ac-8824-6733a54f41b0 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:43.041862 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1239/2485] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:42 2022] DELETE /identity/v3/projects/68c5e7a820a04dcd9230b68fed62eac9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:43.058547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f9620-fbe5-44ac-8824-6733a54f41b0 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:43.186035 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971c79e-2480-4ff0-a118-ec69346f2813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:43.187785 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971c79e-2480-4ff0-a118-ec69346f2813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:43.188272 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971c79e-2480-4ff0-a118-ec69346f2813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:43.188875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971c79e-2480-4ff0-a118-ec69346f2813 admin admin] PATH_INFO: `/v3/users/4a7cd4321e5746e1a9166794e293a5ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:43.191568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971c79e-2480-4ff0-a118-ec69346f2813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a7cd4321e5746e1a9166794e293a5ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:43.194134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971c79e-2480-4ff0-a118-ec69346f2813 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:43.294606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5971c79e-2480-4ff0-a118-ec69346f2813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a7cd4321e5746e1a9166794e293a5ac'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:43.331381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:43.333754 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5971c79e-2480-4ff0-a118-ec69346f2813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a7cd4321e5746e1a9166794e293a5ac'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:43.334764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5971c79e-2480-4ff0-a118-ec69346f2813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a7cd4321e5746e1a9166794e293a5ac'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:43.347811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5971c79e-2480-4ff0-a118-ec69346f2813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a7cd4321e5746e1a9166794e293a5ac'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:43.362673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5971c79e-2480-4ff0-a118-ec69346f2813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a7cd4321e5746e1a9166794e293a5ac'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:43.380143 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1247/2486] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:43 2022] DELETE /identity/v3/users/4a7cd4321e5746e1a9166794e293a5ac => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:44.015462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449754f-5612-47e9-bb51-20f7acc247c2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:44.016427 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449754f-5612-47e9-bb51-20f7acc247c2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:44.016980 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449754f-5612-47e9-bb51-20f7acc247c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:44.029570 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971c79e-2480-4ff0-a118-ec69346f2813 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:44.031115 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5971c79e-2480-4ff0-a118-ec69346f2813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:44.063779 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7449754f-5612-47e9-bb51-20f7acc247c2 None None] MFA Rules not processed for user `538d552b04a8446094d2e84952804795`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:44.065061 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7449754f-5612-47e9-bb51-20f7acc247c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:44.146674 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971c79e-2480-4ff0-a118-ec69346f2813 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:44.156095 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80ded8-2bb5-4112-a06e-546a7caf7087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:44.156825 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80ded8-2bb5-4112-a06e-546a7caf7087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:44.157113 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80ded8-2bb5-4112-a06e-546a7caf7087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:44.157386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80ded8-2bb5-4112-a06e-546a7caf7087 admin admin] PATH_INFO: `/v3/projects/f1251e7ec64b44378ec3f90d7d60b31a` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:44.158593 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80ded8-2bb5-4112-a06e-546a7caf7087 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1251e7ec64b44378ec3f90d7d60b31a)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:44.160501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80ded8-2bb5-4112-a06e-546a7caf7087 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:44.173784 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1240/2487] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:44.179923 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449754f-5612-47e9-bb51-20f7acc247c2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:44.187086 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7449754f-5612-47e9-bb51-20f7acc247c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:44.210337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449754f-5612-47e9-bb51-20f7acc247c2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:44.220176 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-fb80ded8-2bb5-4112-a06e-546a7caf7087 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1251e7ec64b44378ec3f90d7d60b31a', 'request_id': 'req-fb80ded8-2bb5-4112-a06e-546a7caf7087'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:44.229890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-fb80ded8-2bb5-4112-a06e-546a7caf7087 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1251e7ec64b44378ec3f90d7d60b31a', 'request_id': 'req-fb80ded8-2bb5-4112-a06e-546a7caf7087'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:44.237232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-fb80ded8-2bb5-4112-a06e-546a7caf7087 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1251e7ec64b44378ec3f90d7d60b31a', 'request_id': 'req-fb80ded8-2bb5-4112-a06e-546a7caf7087'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:44.251333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-fb80ded8-2bb5-4112-a06e-546a7caf7087 admin admin] The token cache is being invalidate because project f1251e7ec64b44378ec3f90d7d60b31a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:44.251630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-fb80ded8-2bb5-4112-a06e-546a7caf7087 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:44.252326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-fb80ded8-2bb5-4112-a06e-546a7caf7087 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:44.254010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1248/2488] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:44 2022] DELETE /identity/v3/projects/f1251e7ec64b44378ec3f90d7d60b31a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:44.262735 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124b442-906f-4ae5-a36c-70806b94970d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:44.263368 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124b442-906f-4ae5-a36c-70806b94970d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:44.263368 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124b442-906f-4ae5-a36c-70806b94970d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:44.286831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7124b442-906f-4ae5-a36c-70806b94970d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:44.288165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7124b442-906f-4ae5-a36c-70806b94970d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:44.296123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132f53e-8ae4-4dbf-a2d4-c93363749c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:44.296847 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132f53e-8ae4-4dbf-a2d4-c93363749c6c admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:44.297134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132f53e-8ae4-4dbf-a2d4-c93363749c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:44.297418 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132f53e-8ae4-4dbf-a2d4-c93363749c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:44.298138 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132f53e-8ae4-4dbf-a2d4-c93363749c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:44.300240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132f53e-8ae4-4dbf-a2d4-c93363749c6c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:44.331657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1241/2489] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:44 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:44.336887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132f53e-8ae4-4dbf-a2d4-c93363749c6c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:44.338935 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b132f53e-8ae4-4dbf-a2d4-c93363749c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:44.363456 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1249/2490] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:44.374746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccaff1-ec9a-4e63-990b-757733933e5a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:44.374746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccaff1-ec9a-4e63-990b-757733933e5a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:44.374746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccaff1-ec9a-4e63-990b-757733933e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:44.404468 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-36ccaff1-ec9a-4e63-990b-757733933e5a None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:44.406247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-36ccaff1-ec9a-4e63-990b-757733933e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:44.418116 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:44.419606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:44.443539 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132f53e-8ae4-4dbf-a2d4-c93363749c6c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:44.451258 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc00552-2299-44d4-9723-7f59be659bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:44.452068 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc00552-2299-44d4-9723-7f59be659bac admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:44.452374 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc00552-2299-44d4-9723-7f59be659bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:44.452610 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc00552-2299-44d4-9723-7f59be659bac admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:44.453297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc00552-2299-44d4-9723-7f59be659bac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:44.455446 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc00552-2299-44d4-9723-7f59be659bac admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:44.490488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1242/2491] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:44 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:44.495729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc00552-2299-44d4-9723-7f59be659bac admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:44.510983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc00552-2299-44d4-9723-7f59be659bac admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:44.522503 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c567e8e-dfb5-421c-94c7-33875bb09967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:44.523213 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c567e8e-dfb5-421c-94c7-33875bb09967 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:44.523507 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c567e8e-dfb5-421c-94c7-33875bb09967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:44.523762 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c567e8e-dfb5-421c-94c7-33875bb09967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:44.526828 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c567e8e-dfb5-421c-94c7-33875bb09967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:44.528376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c567e8e-dfb5-421c-94c7-33875bb09967 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:44.559267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1243/2492] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:44.561207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c567e8e-dfb5-421c-94c7-33875bb09967 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:44.569093 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1250/2493] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:44.573912 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccaff1-ec9a-4e63-990b-757733933e5a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:44.587932 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c567e8e-dfb5-421c-94c7-33875bb09967 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:44.591624 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccaff1-ec9a-4e63-990b-757733933e5a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:44.599695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d57f1-1620-411e-8436-bbde7b30f9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:44.600430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e2051-8853-42da-b869-b73f60dd7446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:44.601073 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e2051-8853-42da-b869-b73f60dd7446 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:44.601357 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e2051-8853-42da-b869-b73f60dd7446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:44.601674 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e2051-8853-42da-b869-b73f60dd7446 admin admin] PATH_INFO: `/v3/projects/30136a0a94d2491f8621ad2f56caa5a7/users/14be2a82b6e94e3b8d815bfed390faa3/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:44.603870 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d57f1-1620-411e-8436-bbde7b30f9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:44.604512 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d57f1-1620-411e-8436-bbde7b30f9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:44.604512 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d57f1-1620-411e-8436-bbde7b30f9a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:44.605123 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d57f1-1620-411e-8436-bbde7b30f9a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:50:44.605428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d57f1-1620-411e-8436-bbde7b30f9a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:44.607112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d57f1-1620-411e-8436-bbde7b30f9a4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:44.629760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1251/2494] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:50:44.634607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d57f1-1620-411e-8436-bbde7b30f9a4 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:44.635869 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e2051-8853-42da-b869-b73f60dd7446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30136a0a94d2491f8621ad2f56caa5a7, user_id=14be2a82b6e94e3b8d815bfed390faa3, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:44.642800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e2051-8853-42da-b869-b73f60dd7446 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:44.654248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d57f1-1620-411e-8436-bbde7b30f9a4 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:44.663069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a2e20-28ec-4c82-9bb3-086c611d1fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:44.664187 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a2e20-28ec-4c82-9bb3-086c611d1fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:44.665520 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a2e20-28ec-4c82-9bb3-086c611d1fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:44.666038 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a2e20-28ec-4c82-9bb3-086c611d1fb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:44.666925 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a2e20-28ec-4c82-9bb3-086c611d1fb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:44.669676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a2e20-28ec-4c82-9bb3-086c611d1fb4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:44.673546 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1244/2495] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:44 2022] PUT /identity/v3/projects/30136a0a94d2491f8621ad2f56caa5a7/users/14be2a82b6e94e3b8d815bfed390faa3/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:44.676400 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e2051-8853-42da-b869-b73f60dd7446 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:44.739017 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1252/2496] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:44 2022] POST /identity/v3/projects => generated 399 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:44.740572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a2e20-28ec-4c82-9bb3-086c611d1fb4 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:44.792049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e2051-8853-42da-b869-b73f60dd7446 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:44.882627 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a2e20-28ec-4c82-9bb3-086c611d1fb4 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:44.889275 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1b0c3-5e45-4102-aea5-de3b20d0801c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:44.889958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1b0c3-5e45-4102-aea5-de3b20d0801c admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:44.890243 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1b0c3-5e45-4102-aea5-de3b20d0801c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:44.890522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1b0c3-5e45-4102-aea5-de3b20d0801c admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:44.891202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1b0c3-5e45-4102-aea5-de3b20d0801c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:44.893149 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1b0c3-5e45-4102-aea5-de3b20d0801c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:44.897037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4852e-a4e4-4f65-a68f-5cdd0dd3e9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:44.897717 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4852e-a4e4-4f65-a68f-5cdd0dd3e9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:44.897998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4852e-a4e4-4f65-a68f-5cdd0dd3e9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:44.898284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4852e-a4e4-4f65-a68f-5cdd0dd3e9b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:44.899592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4852e-a4e4-4f65-a68f-5cdd0dd3e9b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:44.901202 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4852e-a4e4-4f65-a68f-5cdd0dd3e9b4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:44.916286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1245/2497] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:44.921270 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1253/2498] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:44 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:44.921503 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4852e-a4e4-4f65-a68f-5cdd0dd3e9b4 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:44.922453 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1b0c3-5e45-4102-aea5-de3b20d0801c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:44.930649 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4852e-a4e4-4f65-a68f-5cdd0dd3e9b4 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:44.934070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1b0c3-5e45-4102-aea5-de3b20d0801c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:44.936083 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d94343-514e-4c62-af9a-8dca885309d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:44.937398 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d94343-514e-4c62-af9a-8dca885309d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:44.937740 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d94343-514e-4c62-af9a-8dca885309d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:44.937990 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d94343-514e-4c62-af9a-8dca885309d4 admin admin] PATH_INFO: `/v3/projects/30136a0a94d2491f8621ad2f56caa5a7/users/14be2a82b6e94e3b8d815bfed390faa3/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:44.939219 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc9a3d-ae13-4c33-a871-e616c7ccc645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:44.939842 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc9a3d-ae13-4c33-a871-e616c7ccc645 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:44.940119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc9a3d-ae13-4c33-a871-e616c7ccc645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:44.940277 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d94343-514e-4c62-af9a-8dca885309d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30136a0a94d2491f8621ad2f56caa5a7, user_id=14be2a82b6e94e3b8d815bfed390faa3, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:44.940462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc9a3d-ae13-4c33-a871-e616c7ccc645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:44.941083 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc9a3d-ae13-4c33-a871-e616c7ccc645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:44.942700 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc9a3d-ae13-4c33-a871-e616c7ccc645 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:44.942913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d94343-514e-4c62-af9a-8dca885309d4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:44.958708 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1246/2499] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:44 2022] PUT /identity/v3/projects/30136a0a94d2491f8621ad2f56caa5a7/users/14be2a82b6e94e3b8d815bfed390faa3/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:44.963833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1254/2500] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:50:44.964707 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d94343-514e-4c62-af9a-8dca885309d4 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:45.013424 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:45.041886 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d94343-514e-4c62-af9a-8dca885309d4 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:45.052860 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9e083-a7fe-4760-9546-06dac3db7c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:45.053743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9e083-a7fe-4760-9546-06dac3db7c60 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:45.054393 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9e083-a7fe-4760-9546-06dac3db7c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:45.054829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9e083-a7fe-4760-9546-06dac3db7c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:45.055808 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9e083-a7fe-4760-9546-06dac3db7c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:45.058043 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9e083-a7fe-4760-9546-06dac3db7c60 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:45.071968 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1247/2501] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:50:45.076327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc9a3d-ae13-4c33-a871-e616c7ccc645 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:45.089169 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc9a3d-ae13-4c33-a871-e616c7ccc645 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:45.094427 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719dfdc-3730-4a8e-960a-860795dc1648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:45.095124 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719dfdc-3730-4a8e-960a-860795dc1648 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:45.095405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719dfdc-3730-4a8e-960a-860795dc1648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:45.095682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719dfdc-3730-4a8e-960a-860795dc1648 admin admin] PATH_INFO: `/v3/projects/e05585ddddf9423884592bcc2726df7a/users/0209712aec1349b6857ec47780145be1/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:45.117549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c719dfdc-3730-4a8e-960a-860795dc1648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e05585ddddf9423884592bcc2726df7a, user_id=0209712aec1349b6857ec47780145be1, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:45.121228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c719dfdc-3730-4a8e-960a-860795dc1648 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:45.150312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1255/2502] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:45 2022] PUT /identity/v3/projects/e05585ddddf9423884592bcc2726df7a/users/0209712aec1349b6857ec47780145be1/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:50:45.160993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b8feb-b5ca-4b3b-8a30-7c0f1ebe17af None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:45.161284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b8feb-b5ca-4b3b-8a30-7c0f1ebe17af None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:45.161639 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b8feb-b5ca-4b3b-8a30-7c0f1ebe17af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:45.199431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f02b8feb-b5ca-4b3b-8a30-7c0f1ebe17af None None] MFA Rules not processed for user `0209712aec1349b6857ec47780145be1`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:45.202621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f02b8feb-b5ca-4b3b-8a30-7c0f1ebe17af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:45.341112 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1248/2503] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:45 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:45.346208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719dfdc-3730-4a8e-960a-860795dc1648 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:45.400703 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:45.454670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719dfdc-3730-4a8e-960a-860795dc1648 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:45.462613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fae9bf-681b-421b-8615-69b04723843d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:45.463032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fae9bf-681b-421b-8615-69b04723843d admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:45.468435 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fae9bf-681b-421b-8615-69b04723843d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:45.469290 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fae9bf-681b-421b-8615-69b04723843d admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:45.470056 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fae9bf-681b-421b-8615-69b04723843d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:45.472490 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fae9bf-681b-421b-8615-69b04723843d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:45.524342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1256/2504] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:45 2022] POST /identity/v3/projects => generated 399 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:45.525486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b8feb-b5ca-4b3b-8a30-7c0f1ebe17af None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:45.536894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b8feb-b5ca-4b3b-8a30-7c0f1ebe17af None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:45.603593 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ce804-8cc3-464c-999d-2dcd55e07d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:45.604340 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ce804-8cc3-464c-999d-2dcd55e07d99 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:45.604762 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ce804-8cc3-464c-999d-2dcd55e07d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:45.605028 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ce804-8cc3-464c-999d-2dcd55e07d99 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:45.606071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ce804-8cc3-464c-999d-2dcd55e07d99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:45.608621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ce804-8cc3-464c-999d-2dcd55e07d99 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:45.651723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1249/2505] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:45 2022] POST /identity/v3/users => generated 306 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:45.655736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fae9bf-681b-421b-8615-69b04723843d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:45.667893 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fae9bf-681b-421b-8615-69b04723843d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:45.675109 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025254ca-83e6-49d0-9194-da4680a71fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:45.676244 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025254ca-83e6-49d0-9194-da4680a71fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:45.676591 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025254ca-83e6-49d0-9194-da4680a71fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:45.677034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025254ca-83e6-49d0-9194-da4680a71fc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:45.678064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025254ca-83e6-49d0-9194-da4680a71fc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:45.681305 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025254ca-83e6-49d0-9194-da4680a71fc2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:45.716399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1257/2506] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:45.719585 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ce804-8cc3-464c-999d-2dcd55e07d99 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:45.732288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ce804-8cc3-464c-999d-2dcd55e07d99 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:45.738526 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851ce85-9611-467b-b7a4-bac600032473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:45.739948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851ce85-9611-467b-b7a4-bac600032473 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:45.740242 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851ce85-9611-467b-b7a4-bac600032473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:45.740772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851ce85-9611-467b-b7a4-bac600032473 admin admin] PATH_INFO: `/v3/projects/6d67265624fd4ba89ba3a0a0b7cdaf49/users/1174e11e81a04ba79c0087cd15fdf199/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:45.772728 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851ce85-9611-467b-b7a4-bac600032473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d67265624fd4ba89ba3a0a0b7cdaf49, user_id=1174e11e81a04ba79c0087cd15fdf199, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:45.775559 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851ce85-9611-467b-b7a4-bac600032473 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:45.792536 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1250/2507] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:45 2022] PUT /identity/v3/projects/6d67265624fd4ba89ba3a0a0b7cdaf49/users/1174e11e81a04ba79c0087cd15fdf199/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:50:45.796861 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025254ca-83e6-49d0-9194-da4680a71fc2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:45.808375 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025254ca-83e6-49d0-9194-da4680a71fc2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:45.891476 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8873b88-2fbb-4cd5-8251-f5459c6276f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:45.892018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8873b88-2fbb-4cd5-8251-f5459c6276f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:45.892348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8873b88-2fbb-4cd5-8251-f5459c6276f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:45.892634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8873b88-2fbb-4cd5-8251-f5459c6276f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:45.893335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8873b88-2fbb-4cd5-8251-f5459c6276f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:45.894951 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8873b88-2fbb-4cd5-8251-f5459c6276f9 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:45.905983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1258/2508] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:45.910227 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851ce85-9611-467b-b7a4-bac600032473 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:45.933774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851ce85-9611-467b-b7a4-bac600032473 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:45.938996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1da02-d690-4407-b83f-c1fbf696c558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:45.939704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1da02-d690-4407-b83f-c1fbf696c558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:45.939989 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1da02-d690-4407-b83f-c1fbf696c558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:45.940271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1da02-d690-4407-b83f-c1fbf696c558 admin admin] PATH_INFO: `/v3/projects/6d67265624fd4ba89ba3a0a0b7cdaf49/users/1174e11e81a04ba79c0087cd15fdf199/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:45.942360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1da02-d690-4407-b83f-c1fbf696c558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d67265624fd4ba89ba3a0a0b7cdaf49, user_id=1174e11e81a04ba79c0087cd15fdf199, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:45.945320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1da02-d690-4407-b83f-c1fbf696c558 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:45.958751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1251/2509] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:45 2022] PUT /identity/v3/projects/6d67265624fd4ba89ba3a0a0b7cdaf49/users/1174e11e81a04ba79c0087cd15fdf199/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:45.970789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d75d3-2946-481b-bea5-465ab7ba3fe3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:45.970925 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d75d3-2946-481b-bea5-465ab7ba3fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:45.971084 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d75d3-2946-481b-bea5-465ab7ba3fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:46.012398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-414d75d3-2946-481b-bea5-465ab7ba3fe3 None None] MFA Rules not processed for user `1174e11e81a04ba79c0087cd15fdf199`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:46.013673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-414d75d3-2946-481b-bea5-465ab7ba3fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:46.098286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1259/2510] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:45 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:46.110702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1da02-d690-4407-b83f-c1fbf696c558 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:46.120771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4ed1da02-d690-4407-b83f-c1fbf696c558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:46.142125 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:46.203129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1da02-d690-4407-b83f-c1fbf696c558 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:46.207111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44d09e-db2a-4714-8a61-020d31ae1a06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:46.207773 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44d09e-db2a-4714-8a61-020d31ae1a06 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:46.208057 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44d09e-db2a-4714-8a61-020d31ae1a06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:46.208335 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44d09e-db2a-4714-8a61-020d31ae1a06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:46.219757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44d09e-db2a-4714-8a61-020d31ae1a06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:46.222016 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44d09e-db2a-4714-8a61-020d31ae1a06 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:46.233407 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1252/2511] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:46 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:46.373394 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d75d3-2946-481b-bea5-465ab7ba3fe3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:46.390430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d75d3-2946-481b-bea5-465ab7ba3fe3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:46.395473 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27dbd7-c6ec-40e1-9a30-3827faddf399 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:46.396293 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27dbd7-c6ec-40e1-9a30-3827faddf399 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:46.396693 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27dbd7-c6ec-40e1-9a30-3827faddf399 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:46.397049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27dbd7-c6ec-40e1-9a30-3827faddf399 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:46.447475 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:46.516468 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27dbd7-c6ec-40e1-9a30-3827faddf399 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:46.518301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27dbd7-c6ec-40e1-9a30-3827faddf399 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:46.545147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1260/2512] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:46 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:48.029808 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cf1fd-7faa-403c-8597-db74e9e574a1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:48.030826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cf1fd-7faa-403c-8597-db74e9e574a1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:48.031320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cf1fd-7faa-403c-8597-db74e9e574a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:48.084811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8e3cf1fd-7faa-403c-8597-db74e9e574a1 None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:48.088404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8e3cf1fd-7faa-403c-8597-db74e9e574a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:48.247053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1253/2513] 10.222.0.47 () {58 vars in 967 bytes} [Wed Aug 3 10:50:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:48.256389 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27dbd7-c6ec-40e1-9a30-3827faddf399 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:48.260070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ca27dbd7-c6ec-40e1-9a30-3827faddf399 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:48.478690 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27dbd7-c6ec-40e1-9a30-3827faddf399 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:48.488487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdfe97-665a-47c8-af2c-5b539718065e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:48.489287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdfe97-665a-47c8-af2c-5b539718065e service swift] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:48.489420 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdfe97-665a-47c8-af2c-5b539718065e service swift] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:48.490011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdfe97-665a-47c8-af2c-5b539718065e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:48.503536 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdfe97-665a-47c8-af2c-5b539718065e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:48.510141 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdfe97-665a-47c8-af2c-5b539718065e service swift] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:48.534248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1261/2514] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:50:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 10:50:49.504939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cf1fd-7faa-403c-8597-db74e9e574a1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:49.507701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8e3cf1fd-7faa-403c-8597-db74e9e574a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:49.593446 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cf1fd-7faa-403c-8597-db74e9e574a1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:49.598712 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea72049-29fa-4d4d-90cd-29ad0514df61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:49.599644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea72049-29fa-4d4d-90cd-29ad0514df61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:49.600111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea72049-29fa-4d4d-90cd-29ad0514df61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:49.600814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea72049-29fa-4d4d-90cd-29ad0514df61 admin admin] PATH_INFO: `/v3/users/88cfa5e4719a4eeb94ac3c2175f680ed` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:49.603038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea72049-29fa-4d4d-90cd-29ad0514df61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88cfa5e4719a4eeb94ac3c2175f680ed)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:49.605438 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea72049-29fa-4d4d-90cd-29ad0514df61 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:49.728832 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3ea72049-29fa-4d4d-90cd-29ad0514df61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88cfa5e4719a4eeb94ac3c2175f680ed'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:49.740104 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3ea72049-29fa-4d4d-90cd-29ad0514df61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88cfa5e4719a4eeb94ac3c2175f680ed'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:49.755148 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3ea72049-29fa-4d4d-90cd-29ad0514df61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88cfa5e4719a4eeb94ac3c2175f680ed'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:49.756128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3ea72049-29fa-4d4d-90cd-29ad0514df61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88cfa5e4719a4eeb94ac3c2175f680ed'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:49.756843 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3ea72049-29fa-4d4d-90cd-29ad0514df61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88cfa5e4719a4eeb94ac3c2175f680ed'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:49.759430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:49.805915 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1254/2515] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:49 2022] DELETE /identity/v3/users/88cfa5e4719a4eeb94ac3c2175f680ed => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:49.807818 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0d882-1d64-4614-83eb-f212aeed3717 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:49.813329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0d882-1d64-4614-83eb-f212aeed3717 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:49.813650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0d882-1d64-4614-83eb-f212aeed3717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:49.868209 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9fd0d882-1d64-4614-83eb-f212aeed3717 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:49.869814 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9fd0d882-1d64-4614-83eb-f212aeed3717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:50.012444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1262/2516] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:50.020952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea72049-29fa-4d4d-90cd-29ad0514df61 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:50.022822 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3ea72049-29fa-4d4d-90cd-29ad0514df61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:50.127467 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea72049-29fa-4d4d-90cd-29ad0514df61 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:50.131388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a3c38-e9d5-4c9b-8cd4-b3707d6ef054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:50.132035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a3c38-e9d5-4c9b-8cd4-b3707d6ef054 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:50.132327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a3c38-e9d5-4c9b-8cd4-b3707d6ef054 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:50.132606 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a3c38-e9d5-4c9b-8cd4-b3707d6ef054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:50.144004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a3c38-e9d5-4c9b-8cd4-b3707d6ef054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:50.145950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a3c38-e9d5-4c9b-8cd4-b3707d6ef054 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:50.168979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1255/2517] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:50.655532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0d882-1d64-4614-83eb-f212aeed3717 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:50.658252 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9fd0d882-1d64-4614-83eb-f212aeed3717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:50.682137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0d882-1d64-4614-83eb-f212aeed3717 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:50.698510 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339effb-7420-4af3-be95-3d0ee4286576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:50.699900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339effb-7420-4af3-be95-3d0ee4286576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:50.700268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339effb-7420-4af3-be95-3d0ee4286576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:50.701242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339effb-7420-4af3-be95-3d0ee4286576 admin admin] PATH_INFO: `/v3/projects/9559a93e928e48108b818ac3bb239e48` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:50.706151 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339effb-7420-4af3-be95-3d0ee4286576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9559a93e928e48108b818ac3bb239e48)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:50.714246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339effb-7420-4af3-be95-3d0ee4286576 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:50.839374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1339effb-7420-4af3-be95-3d0ee4286576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9559a93e928e48108b818ac3bb239e48', 'request_id': 'req-1339effb-7420-4af3-be95-3d0ee4286576'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:50.841435 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:50.851460 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1339effb-7420-4af3-be95-3d0ee4286576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9559a93e928e48108b818ac3bb239e48', 'request_id': 'req-1339effb-7420-4af3-be95-3d0ee4286576'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:50.872609 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1339effb-7420-4af3-be95-3d0ee4286576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9559a93e928e48108b818ac3bb239e48', 'request_id': 'req-1339effb-7420-4af3-be95-3d0ee4286576'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:50.889549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1339effb-7420-4af3-be95-3d0ee4286576 admin admin] The token cache is being invalidate because project 9559a93e928e48108b818ac3bb239e48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:50.890582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1339effb-7420-4af3-be95-3d0ee4286576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:50.891464 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1339effb-7420-4af3-be95-3d0ee4286576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:50.895068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1263/2518] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:50 2022] DELETE /identity/v3/projects/9559a93e928e48108b818ac3bb239e48 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:50.899768 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5c475-dffd-4348-8d62-f74ec6900868 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:50.900335 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5c475-dffd-4348-8d62-f74ec6900868 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:50.900658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5c475-dffd-4348-8d62-f74ec6900868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:50.926994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e7e5c475-dffd-4348-8d62-f74ec6900868 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:50.928864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e7e5c475-dffd-4348-8d62-f74ec6900868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:50.995312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a41d88-77fb-426c-ae0d-01d9b69a3030 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:50.995602 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a41d88-77fb-426c-ae0d-01d9b69a3030 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:50.996066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a41d88-77fb-426c-ae0d-01d9b69a3030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:51.017635 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1256/2519] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:51.024435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdde01a-8932-4c60-974b-8fe315a33ecf None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:51.024948 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdde01a-8932-4c60-974b-8fe315a33ecf None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:51.025401 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdde01a-8932-4c60-974b-8fe315a33ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:51.035106 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a9a41d88-77fb-426c-ae0d-01d9b69a3030 None None] MFA Rules not processed for user `14be2a82b6e94e3b8d815bfed390faa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:51.037329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a9a41d88-77fb-426c-ae0d-01d9b69a3030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:51.049599 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-efdde01a-8932-4c60-974b-8fe315a33ecf None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:51.051369 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-efdde01a-8932-4c60-974b-8fe315a33ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:51.063053 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1257/2520] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:51.067625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdde01a-8932-4c60-974b-8fe315a33ecf None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:51.078073 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdde01a-8932-4c60-974b-8fe315a33ecf None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:51.083271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745adbe-9a4f-4efe-a368-f873497ef64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:51.084142 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745adbe-9a4f-4efe-a368-f873497ef64e admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:51.084428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745adbe-9a4f-4efe-a368-f873497ef64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:51.084833 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745adbe-9a4f-4efe-a368-f873497ef64e admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:51.085757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745adbe-9a4f-4efe-a368-f873497ef64e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:50:51.086434 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745adbe-9a4f-4efe-a368-f873497ef64e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:51.089420 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745adbe-9a4f-4efe-a368-f873497ef64e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:51.109292 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1258/2521] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:50:51.115294 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745adbe-9a4f-4efe-a368-f873497ef64e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:51.131449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1264/2522] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:51.146886 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a41d88-77fb-426c-ae0d-01d9b69a3030 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:51.149501 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a9a41d88-77fb-426c-ae0d-01d9b69a3030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:51.152156 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745adbe-9a4f-4efe-a368-f873497ef64e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:51.157756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8714f-60fa-4f0b-add9-f4a1da875b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:51.158576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8714f-60fa-4f0b-add9-f4a1da875b18 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:51.158931 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8714f-60fa-4f0b-add9-f4a1da875b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:51.159320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8714f-60fa-4f0b-add9-f4a1da875b18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:51.160177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8714f-60fa-4f0b-add9-f4a1da875b18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:51.162157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8714f-60fa-4f0b-add9-f4a1da875b18 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:51.196877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1259/2523] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:51 2022] POST /identity/v3/projects => generated 375 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:51.201482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8714f-60fa-4f0b-add9-f4a1da875b18 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:51.237706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a41d88-77fb-426c-ae0d-01d9b69a3030 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:51.259983 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:51.285643 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8714f-60fa-4f0b-add9-f4a1da875b18 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:51.291706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efff2ba-1dfc-48f3-83ba-7c9799c530b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:51.292329 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efff2ba-1dfc-48f3-83ba-7c9799c530b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:51.292711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efff2ba-1dfc-48f3-83ba-7c9799c530b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:51.293179 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efff2ba-1dfc-48f3-83ba-7c9799c530b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:51.293747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efff2ba-1dfc-48f3-83ba-7c9799c530b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:51.300322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09695d30-db78-471b-a4b0-0efe3cb30843 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '300fb5e1c5c546808b935871673e3d0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:51.301041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09695d30-db78-471b-a4b0-0efe3cb30843 service glance] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:51.301562 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09695d30-db78-471b-a4b0-0efe3cb30843 service glance] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:51.301900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09695d30-db78-471b-a4b0-0efe3cb30843 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:51.307955 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-09695d30-db78-471b-a4b0-0efe3cb30843 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:51.308638 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efff2ba-1dfc-48f3-83ba-7c9799c530b8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:51.357511 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1260/2524] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:51 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:51.358921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efff2ba-1dfc-48f3-83ba-7c9799c530b8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:51.363200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:51.370557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efff2ba-1dfc-48f3-83ba-7c9799c530b8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:51.378910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8f3cb-864f-4314-b3b6-df5e2d74e3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:51.380420 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8f3cb-864f-4314-b3b6-df5e2d74e3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:51.380707 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8f3cb-864f-4314-b3b6-df5e2d74e3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:51.381132 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8f3cb-864f-4314-b3b6-df5e2d74e3a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:51.382041 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8f3cb-864f-4314-b3b6-df5e2d74e3a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:51.383650 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8f3cb-864f-4314-b3b6-df5e2d74e3a3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:51.395308 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1261/2525] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:50:51.400306 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8f3cb-864f-4314-b3b6-df5e2d74e3a3 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:51.406947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09695d30-db78-471b-a4b0-0efe3cb30843 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:51.408607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09695d30-db78-471b-a4b0-0efe3cb30843 service glance] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:51.411552 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8f3cb-864f-4314-b3b6-df5e2d74e3a3 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:51.418272 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b3171-d295-4c0e-9ac3-8880d6482f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:51.418946 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b3171-d295-4c0e-9ac3-8880d6482f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:51.419224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b3171-d295-4c0e-9ac3-8880d6482f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:51.419498 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b3171-d295-4c0e-9ac3-8880d6482f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:51.420128 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1265/2526] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:50:51.420363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b3171-d295-4c0e-9ac3-8880d6482f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:51.422048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b3171-d295-4c0e-9ac3-8880d6482f9f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:51.436685 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1262/2527] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:51.447424 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09695d30-db78-471b-a4b0-0efe3cb30843 service glance] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:51.462769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09695d30-db78-471b-a4b0-0efe3cb30843 service glance] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:51.472491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32147ede-45c7-44a8-a5e0-c90c8a0136c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:51.481249 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32147ede-45c7-44a8-a5e0-c90c8a0136c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:51.482031 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32147ede-45c7-44a8-a5e0-c90c8a0136c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:51.482164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32147ede-45c7-44a8-a5e0-c90c8a0136c0 admin admin] PATH_INFO: `/v3/projects/ee44e977cb7e4a2d807ad695302472bf/users/e97a9a7a3172412f9d67af6537670016/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:51.512573 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32147ede-45c7-44a8-a5e0-c90c8a0136c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee44e977cb7e4a2d807ad695302472bf, user_id=e97a9a7a3172412f9d67af6537670016, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:51.515314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32147ede-45c7-44a8-a5e0-c90c8a0136c0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:51.533006 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1266/2528] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:51 2022] PUT /identity/v3/projects/ee44e977cb7e4a2d807ad695302472bf/users/e97a9a7a3172412f9d67af6537670016/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:51.548488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de054983-3a99-4741-9d0b-b14fd2188c05 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:51.549059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de054983-3a99-4741-9d0b-b14fd2188c05 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:51.549340 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de054983-3a99-4741-9d0b-b14fd2188c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:51.582211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-de054983-3a99-4741-9d0b-b14fd2188c05 None None] MFA Rules not processed for user `e97a9a7a3172412f9d67af6537670016`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:51.583753 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-de054983-3a99-4741-9d0b-b14fd2188c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:51.594804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a0939-f947-4036-b8d5-525ea95c4fa4 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:51.595603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a0939-f947-4036-b8d5-525ea95c4fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:51.595603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a0939-f947-4036-b8d5-525ea95c4fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:51.617570 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3b8a0939-f947-4036-b8d5-525ea95c4fa4 None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:51.618846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3b8a0939-f947-4036-b8d5-525ea95c4fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:51.693001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1263/2529] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:51.693913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de054983-3a99-4741-9d0b-b14fd2188c05 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:51.700357 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-de054983-3a99-4741-9d0b-b14fd2188c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:51.704604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1267/2530] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 3 10:50:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:51.706136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a0939-f947-4036-b8d5-525ea95c4fa4 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:51.708059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3b8a0939-f947-4036-b8d5-525ea95c4fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:51.778435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de054983-3a99-4741-9d0b-b14fd2188c05 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:51.784321 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1181077-833a-423b-bcbb-8393469550d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:51.785223 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1181077-833a-423b-bcbb-8393469550d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:51.785532 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1181077-833a-423b-bcbb-8393469550d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:51.785811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1181077-833a-423b-bcbb-8393469550d6 admin admin] PATH_INFO: `/v3/users/0209712aec1349b6857ec47780145be1` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:51.787060 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1181077-833a-423b-bcbb-8393469550d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0209712aec1349b6857ec47780145be1)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:51.789045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1181077-833a-423b-bcbb-8393469550d6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:51.817459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a0939-f947-4036-b8d5-525ea95c4fa4 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:51.825245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddd0e9-e838-489e-a858-a678d7cd5fb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:51.826075 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddd0e9-e838-489e-a858-a678d7cd5fb2 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:51.826433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddd0e9-e838-489e-a858-a678d7cd5fb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:51.827110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddd0e9-e838-489e-a858-a678d7cd5fb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:51.837443 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddd0e9-e838-489e-a858-a678d7cd5fb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:51.839240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddd0e9-e838-489e-a858-a678d7cd5fb2 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:51.850094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1268/2531] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:51.850672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:51.862682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddd0e9-e838-489e-a858-a678d7cd5fb2 service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:51.864227 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-35ddd0e9-e838-489e-a858-a678d7cd5fb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:51.865562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c1181077-833a-423b-bcbb-8393469550d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0209712aec1349b6857ec47780145be1'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:51.880963 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c1181077-833a-423b-bcbb-8393469550d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0209712aec1349b6857ec47780145be1'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:51.915157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c1181077-833a-423b-bcbb-8393469550d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0209712aec1349b6857ec47780145be1'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:51.916004 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c1181077-833a-423b-bcbb-8393469550d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0209712aec1349b6857ec47780145be1'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:51.916735 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c1181077-833a-423b-bcbb-8393469550d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0209712aec1349b6857ec47780145be1'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:51.949992 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1264/2532] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:51 2022] DELETE /identity/v3/users/0209712aec1349b6857ec47780145be1 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:51.952688 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:51.955688 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af719cac-345a-4d92-aa8f-612acaccb406 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:51.956723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af719cac-345a-4d92-aa8f-612acaccb406 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:51.957528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af719cac-345a-4d92-aa8f-612acaccb406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:51.998363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-af719cac-345a-4d92-aa8f-612acaccb406 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:52.000665 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-af719cac-345a-4d92-aa8f-612acaccb406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:52.004175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddd0e9-e838-489e-a858-a678d7cd5fb2 service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:52.088052 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1265/2533] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:52.105876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af719cac-345a-4d92-aa8f-612acaccb406 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:52.107618 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-af719cac-345a-4d92-aa8f-612acaccb406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:52.110301 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d6a059-27b0-436f-8777-8318a48d7e3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:52.111011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6a059-27b0-436f-8777-8318a48d7e3d service swift] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:52.111356 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6a059-27b0-436f-8777-8318a48d7e3d service swift] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:52.111693 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6a059-27b0-436f-8777-8318a48d7e3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:52.113943 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-79d6a059-27b0-436f-8777-8318a48d7e3d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:52.180918 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af719cac-345a-4d92-aa8f-612acaccb406 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:52.184706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499a771-a687-4df0-a335-44baf4e1b96d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:52.185422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d6a059-27b0-436f-8777-8318a48d7e3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:52.185592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499a771-a687-4df0-a335-44baf4e1b96d service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:52.185767 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499a771-a687-4df0-a335-44baf4e1b96d service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:52.186024 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499a771-a687-4df0-a335-44baf4e1b96d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:52.187182 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d6a059-27b0-436f-8777-8318a48d7e3d service swift] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:52.197586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1269/2534] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:50:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 10:50:52.198410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d6a059-27b0-436f-8777-8318a48d7e3d service swift] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:52.199981 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-79d6a059-27b0-436f-8777-8318a48d7e3d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:52.203297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499a771-a687-4df0-a335-44baf4e1b96d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:52.204850 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499a771-a687-4df0-a335-44baf4e1b96d service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:52.217455 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d6a059-27b0-436f-8777-8318a48d7e3d service swift] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:52.218831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1266/2535] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:52.241191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b02ad1-04bc-4e87-950b-36c80ebf6aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:52.242019 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b02ad1-04bc-4e87-950b-36c80ebf6aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:52.242199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b02ad1-04bc-4e87-950b-36c80ebf6aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:52.242437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b02ad1-04bc-4e87-950b-36c80ebf6aee admin admin] PATH_INFO: `/v3/users/e97a9a7a3172412f9d67af6537670016` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:52.245735 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b02ad1-04bc-4e87-950b-36c80ebf6aee admin admin] RBAC: Authorizing `identity:delete_user(user_id=e97a9a7a3172412f9d67af6537670016)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:52.247751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b02ad1-04bc-4e87-950b-36c80ebf6aee admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:52.375861 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d6b02ad1-04bc-4e87-950b-36c80ebf6aee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e97a9a7a3172412f9d67af6537670016'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:52.402593 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d6b02ad1-04bc-4e87-950b-36c80ebf6aee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e97a9a7a3172412f9d67af6537670016'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:52.403496 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d6b02ad1-04bc-4e87-950b-36c80ebf6aee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e97a9a7a3172412f9d67af6537670016'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:52.417588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d6b02ad1-04bc-4e87-950b-36c80ebf6aee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e97a9a7a3172412f9d67af6537670016'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:52.440099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:52.445142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d6b02ad1-04bc-4e87-950b-36c80ebf6aee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e97a9a7a3172412f9d67af6537670016'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:52.473547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1270/2536] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:52 2022] DELETE /identity/v3/users/e97a9a7a3172412f9d67af6537670016 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:52.478574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777b200-f2cf-4c7d-800c-6c696dc7a69f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:52.479091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777b200-f2cf-4c7d-800c-6c696dc7a69f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:52.479752 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777b200-f2cf-4c7d-800c-6c696dc7a69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:52.507313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4777b200-f2cf-4c7d-800c-6c696dc7a69f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:52.508601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4777b200-f2cf-4c7d-800c-6c696dc7a69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:52.616459 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1267/2537] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:52.629401 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b02ad1-04bc-4e87-950b-36c80ebf6aee admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:52.631640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d6b02ad1-04bc-4e87-950b-36c80ebf6aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:52.773017 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b02ad1-04bc-4e87-950b-36c80ebf6aee admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:52.782439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46ab8f-1933-4260-a091-634de0fcae1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:52.786555 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46ab8f-1933-4260-a091-634de0fcae1f service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:52.794839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46ab8f-1933-4260-a091-634de0fcae1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:52.795562 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46ab8f-1933-4260-a091-634de0fcae1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:52.817186 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46ab8f-1933-4260-a091-634de0fcae1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:52.819619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46ab8f-1933-4260-a091-634de0fcae1f service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:52.867340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1271/2538] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:52.868333 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777b200-f2cf-4c7d-800c-6c696dc7a69f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:52.869952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4777b200-f2cf-4c7d-800c-6c696dc7a69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:52.878431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:52.885806 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777b200-f2cf-4c7d-800c-6c696dc7a69f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:52.896077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:52.897042 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:52.902267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:52.902680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5 admin admin] PATH_INFO: `/v3/projects/e05585ddddf9423884592bcc2726df7a` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:52.904205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e05585ddddf9423884592bcc2726df7a)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:52.906416 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:53.008102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e05585ddddf9423884592bcc2726df7a', 'request_id': 'req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:53.020967 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e05585ddddf9423884592bcc2726df7a', 'request_id': 'req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:53.033363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e05585ddddf9423884592bcc2726df7a', 'request_id': 'req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:53.067074 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5 admin admin] The token cache is being invalidate because project e05585ddddf9423884592bcc2726df7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:53.067510 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:53.070633 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:53.075217 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1268/2539] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:52 2022] DELETE /identity/v3/projects/e05585ddddf9423884592bcc2726df7a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:50:53.079193 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46ab8f-1933-4260-a091-634de0fcae1f service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:53.090279 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46ab8f-1933-4260-a091-634de0fcae1f service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:53.159105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00339b91-91cd-44cc-98a0-eda38df6b59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:53.159486 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00339b91-91cd-44cc-98a0-eda38df6b59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:53.159878 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00339b91-91cd-44cc-98a0-eda38df6b59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:53.162525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00339b91-91cd-44cc-98a0-eda38df6b59e admin admin] PATH_INFO: `/v3/users/1174e11e81a04ba79c0087cd15fdf199` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:53.162525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00339b91-91cd-44cc-98a0-eda38df6b59e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1174e11e81a04ba79c0087cd15fdf199)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:53.164213 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00339b91-91cd-44cc-98a0-eda38df6b59e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:53.228767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-00339b91-91cd-44cc-98a0-eda38df6b59e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1174e11e81a04ba79c0087cd15fdf199'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:53.245302 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-00339b91-91cd-44cc-98a0-eda38df6b59e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1174e11e81a04ba79c0087cd15fdf199'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:53.248669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-00339b91-91cd-44cc-98a0-eda38df6b59e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1174e11e81a04ba79c0087cd15fdf199'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:53.261291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-00339b91-91cd-44cc-98a0-eda38df6b59e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1174e11e81a04ba79c0087cd15fdf199'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:53.271120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-00339b91-91cd-44cc-98a0-eda38df6b59e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1174e11e81a04ba79c0087cd15fdf199'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:53.285051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1272/2540] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:53 2022] DELETE /identity/v3/users/1174e11e81a04ba79c0087cd15fdf199 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:50:53.518594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:53.520439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:53.546562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759ba45-fea5-471b-8dc2-6e44d6f7bdc5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:53.674572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a053fa-00b2-450a-8c32-341ce25b3569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:53.675436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a053fa-00b2-450a-8c32-341ce25b3569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:53.675825 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a053fa-00b2-450a-8c32-341ce25b3569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:53.676224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a053fa-00b2-450a-8c32-341ce25b3569 admin admin] PATH_INFO: `/v3/projects/ee44e977cb7e4a2d807ad695302472bf` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:53.678608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a053fa-00b2-450a-8c32-341ce25b3569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee44e977cb7e4a2d807ad695302472bf)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:53.685049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a053fa-00b2-450a-8c32-341ce25b3569 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:53.774715 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e1a053fa-00b2-450a-8c32-341ce25b3569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee44e977cb7e4a2d807ad695302472bf', 'request_id': 'req-e1a053fa-00b2-450a-8c32-341ce25b3569'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:53.795102 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e1a053fa-00b2-450a-8c32-341ce25b3569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee44e977cb7e4a2d807ad695302472bf', 'request_id': 'req-e1a053fa-00b2-450a-8c32-341ce25b3569'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:53.808237 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e1a053fa-00b2-450a-8c32-341ce25b3569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee44e977cb7e4a2d807ad695302472bf', 'request_id': 'req-e1a053fa-00b2-450a-8c32-341ce25b3569'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:53.827344 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00339b91-91cd-44cc-98a0-eda38df6b59e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:53.829193 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-00339b91-91cd-44cc-98a0-eda38df6b59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:53.841220 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e1a053fa-00b2-450a-8c32-341ce25b3569 admin admin] The token cache is being invalidate because project ee44e977cb7e4a2d807ad695302472bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:53.841555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e1a053fa-00b2-450a-8c32-341ce25b3569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:53.842509 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e1a053fa-00b2-450a-8c32-341ce25b3569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:53.844381 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1269/2541] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:53 2022] DELETE /identity/v3/projects/ee44e977cb7e4a2d807ad695302472bf => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:53.858438 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c50e8e-6285-4d2c-a439-aea4145284ba None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:53.860537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c50e8e-6285-4d2c-a439-aea4145284ba None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:53.862205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c50e8e-6285-4d2c-a439-aea4145284ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:53.868941 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00339b91-91cd-44cc-98a0-eda38df6b59e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:53.905404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:53.906291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c9c50e8e-6285-4d2c-a439-aea4145284ba None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:53.911923 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c9c50e8e-6285-4d2c-a439-aea4145284ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:53.934016 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:54.006385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684eb41-088f-4d92-abec-7c2a93eeb756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:54.007066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684eb41-088f-4d92-abec-7c2a93eeb756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.007365 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684eb41-088f-4d92-abec-7c2a93eeb756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.007633 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684eb41-088f-4d92-abec-7c2a93eeb756 admin admin] PATH_INFO: `/v3/projects/6d67265624fd4ba89ba3a0a0b7cdaf49` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.010991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684eb41-088f-4d92-abec-7c2a93eeb756 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d67265624fd4ba89ba3a0a0b7cdaf49)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:54.012974 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684eb41-088f-4d92-abec-7c2a93eeb756 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:54.031907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1270/2542] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:54.041483 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fea95-4694-4d71-9e18-03f5d0ba1765 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.041995 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fea95-4694-4d71-9e18-03f5d0ba1765 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.042338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fea95-4694-4d71-9e18-03f5d0ba1765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.067901 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6684eb41-088f-4d92-abec-7c2a93eeb756 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d67265624fd4ba89ba3a0a0b7cdaf49', 'request_id': 'req-6684eb41-088f-4d92-abec-7c2a93eeb756'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:54.077621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6684eb41-088f-4d92-abec-7c2a93eeb756 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d67265624fd4ba89ba3a0a0b7cdaf49', 'request_id': 'req-6684eb41-088f-4d92-abec-7c2a93eeb756'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:54.082190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e58fea95-4694-4d71-9e18-03f5d0ba1765 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:54.083910 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e58fea95-4694-4d71-9e18-03f5d0ba1765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:54.085337 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6684eb41-088f-4d92-abec-7c2a93eeb756 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d67265624fd4ba89ba3a0a0b7cdaf49', 'request_id': 'req-6684eb41-088f-4d92-abec-7c2a93eeb756'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:54.097256 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6684eb41-088f-4d92-abec-7c2a93eeb756 admin admin] The token cache is being invalidate because project 6d67265624fd4ba89ba3a0a0b7cdaf49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:54.097592 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6684eb41-088f-4d92-abec-7c2a93eeb756 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:54.098283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6684eb41-088f-4d92-abec-7c2a93eeb756 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:54.100079 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1273/2543] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:53 2022] DELETE /identity/v3/projects/6d67265624fd4ba89ba3a0a0b7cdaf49 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:54.105759 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1eed7-3768-4275-b872-6ba276d97648 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.105940 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1eed7-3768-4275-b872-6ba276d97648 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.106160 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1eed7-3768-4275-b872-6ba276d97648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.127698 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3ce1eed7-3768-4275-b872-6ba276d97648 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:54.129006 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3ce1eed7-3768-4275-b872-6ba276d97648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:54.170279 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1271/2544] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:54.171277 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fea95-4694-4d71-9e18-03f5d0ba1765 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:54.182422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fea95-4694-4d71-9e18-03f5d0ba1765 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:54.188979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c9def-2824-456d-869a-3c493e07c8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:54.189725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c9def-2824-456d-869a-3c493e07c8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.190013 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c9def-2824-456d-869a-3c493e07c8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.190296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c9def-2824-456d-869a-3c493e07c8a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.191021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c9def-2824-456d-869a-3c493e07c8a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:50:54.191332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c9def-2824-456d-869a-3c493e07c8a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:54.193654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c9def-2824-456d-869a-3c493e07c8a5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:54.209214 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1272/2545] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:50:54.213615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c9def-2824-456d-869a-3c493e07c8a5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:54.217530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1274/2546] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:54.227376 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01351ecb-0808-4c94-a947-711288fc5f40 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.227656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01351ecb-0808-4c94-a947-711288fc5f40 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.227928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01351ecb-0808-4c94-a947-711288fc5f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.229157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c9def-2824-456d-869a-3c493e07c8a5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:54.237302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d90cd-4d97-4645-bb20-e58e42cefe14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:54.238464 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d90cd-4d97-4645-bb20-e58e42cefe14 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.238931 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d90cd-4d97-4645-bb20-e58e42cefe14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.239368 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d90cd-4d97-4645-bb20-e58e42cefe14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.240475 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d90cd-4d97-4645-bb20-e58e42cefe14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:54.243705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d90cd-4d97-4645-bb20-e58e42cefe14 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:54.250956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-01351ecb-0808-4c94-a947-711288fc5f40 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:54.252185 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-01351ecb-0808-4c94-a947-711288fc5f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:54.265053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1275/2547] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:54.266234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01351ecb-0808-4c94-a947-711288fc5f40 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:54.279786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01351ecb-0808-4c94-a947-711288fc5f40 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:54.284179 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1273/2548] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:54 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:54.291540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d90cd-4d97-4645-bb20-e58e42cefe14 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:54.291924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc78862-e6ab-4f95-8f69-9a24d8bc0237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:54.292571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc78862-e6ab-4f95-8f69-9a24d8bc0237 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.292854 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc78862-e6ab-4f95-8f69-9a24d8bc0237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.293131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc78862-e6ab-4f95-8f69-9a24d8bc0237 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.293889 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc78862-e6ab-4f95-8f69-9a24d8bc0237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:50:54.294199 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc78862-e6ab-4f95-8f69-9a24d8bc0237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:54.296005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc78862-e6ab-4f95-8f69-9a24d8bc0237 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:54.311925 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1276/2549] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:50:54.316601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc78862-e6ab-4f95-8f69-9a24d8bc0237 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:54.339382 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc78862-e6ab-4f95-8f69-9a24d8bc0237 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:54.376672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d90cd-4d97-4645-bb20-e58e42cefe14 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:54.389966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:54.391660 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1c428-0dbe-4c09-84b9-3e731150c4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:54.392423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1c428-0dbe-4c09-84b9-3e731150c4ef admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.392821 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1c428-0dbe-4c09-84b9-3e731150c4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.393207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1c428-0dbe-4c09-84b9-3e731150c4ef admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.394164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1c428-0dbe-4c09-84b9-3e731150c4ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:54.397318 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1c428-0dbe-4c09-84b9-3e731150c4ef admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:54.425712 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1274/2550] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:54 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:54.429506 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1c428-0dbe-4c09-84b9-3e731150c4ef admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:54.445493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1c428-0dbe-4c09-84b9-3e731150c4ef admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:54.450361 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94fcae-db4e-408e-b7c4-ff9b12c27b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:54.450935 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c11bc-6d23-4e74-b42e-50c1bfd57879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:54.451083 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94fcae-db4e-408e-b7c4-ff9b12c27b57 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.451278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94fcae-db4e-408e-b7c4-ff9b12c27b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.451682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94fcae-db4e-408e-b7c4-ff9b12c27b57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.451942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c11bc-6d23-4e74-b42e-50c1bfd57879 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.452326 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c11bc-6d23-4e74-b42e-50c1bfd57879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.452422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94fcae-db4e-408e-b7c4-ff9b12c27b57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:54.452772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c11bc-6d23-4e74-b42e-50c1bfd57879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.453700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c11bc-6d23-4e74-b42e-50c1bfd57879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:54.454307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94fcae-db4e-408e-b7c4-ff9b12c27b57 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:54.455486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c11bc-6d23-4e74-b42e-50c1bfd57879 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:54.465192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:54.475523 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1275/2551] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:50:54.478424 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c11bc-6d23-4e74-b42e-50c1bfd57879 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:54.490159 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c11bc-6d23-4e74-b42e-50c1bfd57879 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:54.497215 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25793d9e-8be6-4b2a-a4e6-cb741ff37ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:54.497215 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25793d9e-8be6-4b2a-a4e6-cb741ff37ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.497564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25793d9e-8be6-4b2a-a4e6-cb741ff37ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.497659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25793d9e-8be6-4b2a-a4e6-cb741ff37ad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.498338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25793d9e-8be6-4b2a-a4e6-cb741ff37ad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:54.499968 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25793d9e-8be6-4b2a-a4e6-cb741ff37ad4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:54.500646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1277/2552] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:54 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:54.504784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94fcae-db4e-408e-b7c4-ff9b12c27b57 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:54.512435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1276/2553] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:50:54.517291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25793d9e-8be6-4b2a-a4e6-cb741ff37ad4 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:54.530697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25793d9e-8be6-4b2a-a4e6-cb741ff37ad4 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:54.595103 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94fcae-db4e-408e-b7c4-ff9b12c27b57 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:54.605314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bf4a9-f9f7-48ea-b2e3-087386d35631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:54.606522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bf4a9-f9f7-48ea-b2e3-087386d35631 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.606971 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bf4a9-f9f7-48ea-b2e3-087386d35631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.607581 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bf4a9-f9f7-48ea-b2e3-087386d35631 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.609716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bf4a9-f9f7-48ea-b2e3-087386d35631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:54.611997 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bf4a9-f9f7-48ea-b2e3-087386d35631 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:54.616751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac435a-ac15-48ed-a21b-09fed166677d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:54.617783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac435a-ac15-48ed-a21b-09fed166677d admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.618408 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac435a-ac15-48ed-a21b-09fed166677d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.618876 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac435a-ac15-48ed-a21b-09fed166677d admin admin] PATH_INFO: `/v3/projects/eca6fff00cf94e4c929295e58e4a6937/users/770f358a86db45e5ada5c0784cfec364/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.649622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac435a-ac15-48ed-a21b-09fed166677d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eca6fff00cf94e4c929295e58e4a6937, user_id=770f358a86db45e5ada5c0784cfec364, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:54.649622 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac435a-ac15-48ed-a21b-09fed166677d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:54.660545 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1278/2554] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:54 2022] POST /identity/v3/users => generated 290 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:54.666366 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bf4a9-f9f7-48ea-b2e3-087386d35631 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:54.682180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1277/2555] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:54 2022] PUT /identity/v3/projects/eca6fff00cf94e4c929295e58e4a6937/users/770f358a86db45e5ada5c0784cfec364/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:54.693697 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bf4a9-f9f7-48ea-b2e3-087386d35631 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:54.701037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165e63a-d9ea-4e64-8080-a28933f4e76d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.701325 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165e63a-d9ea-4e64-8080-a28933f4e76d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.701658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165e63a-d9ea-4e64-8080-a28933f4e76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.749043 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f165e63a-d9ea-4e64-8080-a28933f4e76d None None] MFA Rules not processed for user `770f358a86db45e5ada5c0784cfec364`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:54.750449 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f165e63a-d9ea-4e64-8080-a28933f4e76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:54.797384 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461391c-4496-47e5-bb07-b2f54b049c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:54.798618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461391c-4496-47e5-bb07-b2f54b049c03 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.798911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461391c-4496-47e5-bb07-b2f54b049c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.799193 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461391c-4496-47e5-bb07-b2f54b049c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.799915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461391c-4496-47e5-bb07-b2f54b049c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:54.802141 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461391c-4496-47e5-bb07-b2f54b049c03 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:54.815862 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1279/2556] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:54.820197 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461391c-4496-47e5-bb07-b2f54b049c03 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:54.836780 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461391c-4496-47e5-bb07-b2f54b049c03 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:54.842016 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a12148-f5cc-401b-8b72-13175a593fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:54.842649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a12148-f5cc-401b-8b72-13175a593fbe admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.842934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a12148-f5cc-401b-8b72-13175a593fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.843238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a12148-f5cc-401b-8b72-13175a593fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.843960 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a12148-f5cc-401b-8b72-13175a593fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:54.845645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a12148-f5cc-401b-8b72-13175a593fbe admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:54.873376 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1278/2557] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:54.878018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165e63a-d9ea-4e64-8080-a28933f4e76d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:54.887121 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1280/2558] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:54.892648 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a12148-f5cc-401b-8b72-13175a593fbe admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:54.897428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165e63a-d9ea-4e64-8080-a28933f4e76d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:54.907367 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2428e-4acb-4f7c-8010-977a7a8d8deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:54.908091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2428e-4acb-4f7c-8010-977a7a8d8deb admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.908336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2428e-4acb-4f7c-8010-977a7a8d8deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.908620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2428e-4acb-4f7c-8010-977a7a8d8deb admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.909399 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2428e-4acb-4f7c-8010-977a7a8d8deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:54.910205 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a12148-f5cc-401b-8b72-13175a593fbe admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:54.911525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2428e-4acb-4f7c-8010-977a7a8d8deb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:54.916202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323294f-96d6-4ae5-842f-cbd618bb71fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:54.916909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323294f-96d6-4ae5-842f-cbd618bb71fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.917202 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323294f-96d6-4ae5-842f-cbd618bb71fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.917495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323294f-96d6-4ae5-842f-cbd618bb71fe admin admin] PATH_INFO: `/v3/projects/0c9f244aebdf4d3e9a379b0bb999fa56/users/1c7f3fb6979b437bbaa8f80930ca6f06/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.937875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323294f-96d6-4ae5-842f-cbd618bb71fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9f244aebdf4d3e9a379b0bb999fa56, user_id=1c7f3fb6979b437bbaa8f80930ca6f06, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:54.941291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323294f-96d6-4ae5-842f-cbd618bb71fe admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:54.953847 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1279/2559] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:54 2022] POST /identity/v3/projects => generated 387 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:54.957419 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2428e-4acb-4f7c-8010-977a7a8d8deb admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:54.969556 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1281/2560] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:54 2022] PUT /identity/v3/projects/0c9f244aebdf4d3e9a379b0bb999fa56/users/1c7f3fb6979b437bbaa8f80930ca6f06/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:54.971803 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9fda7-15d5-4dd6-a3fe-01342a0fcf69 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:54.972160 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9fda7-15d5-4dd6-a3fe-01342a0fcf69 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:54.972503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9fda7-15d5-4dd6-a3fe-01342a0fcf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:54.983055 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:55.014390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-52d9fda7-15d5-4dd6-a3fe-01342a0fcf69 None None] MFA Rules not processed for user `1c7f3fb6979b437bbaa8f80930ca6f06`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:55.015824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-52d9fda7-15d5-4dd6-a3fe-01342a0fcf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:55.033280 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:55.070095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2428e-4acb-4f7c-8010-977a7a8d8deb admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:55.076274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117894c-1332-4523-ac1c-2387e3c25013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:55.076942 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117894c-1332-4523-ac1c-2387e3c25013 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:55.077213 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117894c-1332-4523-ac1c-2387e3c25013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:55.077601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117894c-1332-4523-ac1c-2387e3c25013 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:55.078264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117894c-1332-4523-ac1c-2387e3c25013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:55.080423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117894c-1332-4523-ac1c-2387e3c25013 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:55.107027 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1280/2561] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:54 2022] POST /identity/v3/users => generated 300 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:55.110926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117894c-1332-4523-ac1c-2387e3c25013 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:55.122754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117894c-1332-4523-ac1c-2387e3c25013 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:55.127898 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c662f4-2837-4e7f-9be1-4caa05083fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:55.128510 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c662f4-2837-4e7f-9be1-4caa05083fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:55.128796 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c662f4-2837-4e7f-9be1-4caa05083fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:55.129073 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c662f4-2837-4e7f-9be1-4caa05083fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:55.129933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c662f4-2837-4e7f-9be1-4caa05083fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:55.131562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c662f4-2837-4e7f-9be1-4caa05083fb2 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:55.137380 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1282/2562] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:55.141943 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9fda7-15d5-4dd6-a3fe-01342a0fcf69 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:55.150286 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1281/2563] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:50:55.154349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c662f4-2837-4e7f-9be1-4caa05083fb2 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:55.166374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9fda7-15d5-4dd6-a3fe-01342a0fcf69 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:55.167738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c662f4-2837-4e7f-9be1-4caa05083fb2 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:55.171994 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025749f-2a72-4a0d-827c-161406b5ac9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:55.172629 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025749f-2a72-4a0d-827c-161406b5ac9e admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:55.172917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025749f-2a72-4a0d-827c-161406b5ac9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:55.173208 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025749f-2a72-4a0d-827c-161406b5ac9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:55.173952 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677a1eb-266e-4af3-9229-59368fcc4d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:55.174865 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677a1eb-266e-4af3-9229-59368fcc4d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:55.175325 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677a1eb-266e-4af3-9229-59368fcc4d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:55.175831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677a1eb-266e-4af3-9229-59368fcc4d39 admin admin] PATH_INFO: `/v3/projects/7dc96ce52b734b0ca017ba7792cc1723/users/3aa95b09a72c493fb5d02f407a4918d5/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:55.179514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025749f-2a72-4a0d-827c-161406b5ac9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:55.184723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025749f-2a72-4a0d-827c-161406b5ac9e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:55.201154 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677a1eb-266e-4af3-9229-59368fcc4d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc96ce52b734b0ca017ba7792cc1723, user_id=3aa95b09a72c493fb5d02f407a4918d5, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:55.204233 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677a1eb-266e-4af3-9229-59368fcc4d39 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:55.230543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1282/2564] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:55 2022] PUT /identity/v3/projects/7dc96ce52b734b0ca017ba7792cc1723/users/3aa95b09a72c493fb5d02f407a4918d5/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:55.231903 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677a1eb-266e-4af3-9229-59368fcc4d39 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:55.247918 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1283/2565] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:55 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:55.251821 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025749f-2a72-4a0d-827c-161406b5ac9e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:55.313635 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677a1eb-266e-4af3-9229-59368fcc4d39 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:55.368529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025749f-2a72-4a0d-827c-161406b5ac9e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:55.385864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2327bf7-f226-4c3c-a9c9-83ff023ce4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:55.386463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2327bf7-f226-4c3c-a9c9-83ff023ce4ac admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:55.387350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2327bf7-f226-4c3c-a9c9-83ff023ce4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:55.387350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2327bf7-f226-4c3c-a9c9-83ff023ce4ac admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:55.388032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2327bf7-f226-4c3c-a9c9-83ff023ce4ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:55.392395 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2327bf7-f226-4c3c-a9c9-83ff023ce4ac admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:55.428582 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f44a1-12a4-48df-9407-314ddfc94641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:55.429596 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f44a1-12a4-48df-9407-314ddfc94641 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:55.430640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f44a1-12a4-48df-9407-314ddfc94641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:55.431113 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f44a1-12a4-48df-9407-314ddfc94641 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:55.436262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f44a1-12a4-48df-9407-314ddfc94641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:55.443106 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1284/2566] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:55 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:55.443288 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f44a1-12a4-48df-9407-314ddfc94641 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:55.444987 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2327bf7-f226-4c3c-a9c9-83ff023ce4ac admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:55.468016 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2327bf7-f226-4c3c-a9c9-83ff023ce4ac admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:55.473162 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1283/2567] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:55.477817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f44a1-12a4-48df-9407-314ddfc94641 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:55.484924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b73caa-806b-49a3-989c-bd50422aaa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:55.485850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b73caa-806b-49a3-989c-bd50422aaa4d admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:55.486828 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b73caa-806b-49a3-989c-bd50422aaa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:55.488401 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b73caa-806b-49a3-989c-bd50422aaa4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:55.489274 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b73caa-806b-49a3-989c-bd50422aaa4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:55.495008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b73caa-806b-49a3-989c-bd50422aaa4d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:55.502141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f44a1-12a4-48df-9407-314ddfc94641 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:55.524579 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007572c-056f-497d-9e43-fea033e40981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:55.526139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007572c-056f-497d-9e43-fea033e40981 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:55.528754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007572c-056f-497d-9e43-fea033e40981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:55.529169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007572c-056f-497d-9e43-fea033e40981 admin admin] PATH_INFO: `/v3/projects/7dc96ce52b734b0ca017ba7792cc1723/users/3aa95b09a72c493fb5d02f407a4918d5/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:55.533446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1285/2568] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:55.534711 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b73caa-806b-49a3-989c-bd50422aaa4d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:55.535909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007572c-056f-497d-9e43-fea033e40981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc96ce52b734b0ca017ba7792cc1723, user_id=3aa95b09a72c493fb5d02f407a4918d5, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:55.540463 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007572c-056f-497d-9e43-fea033e40981 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:55.556349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b73caa-806b-49a3-989c-bd50422aaa4d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:55.570297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1284/2569] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:55 2022] PUT /identity/v3/projects/7dc96ce52b734b0ca017ba7792cc1723/users/3aa95b09a72c493fb5d02f407a4918d5/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:55.581083 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c7c31-d429-4efc-9b1d-83046c17a5b8 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:55.581374 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c7c31-d429-4efc-9b1d-83046c17a5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:55.581688 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c7c31-d429-4efc-9b1d-83046c17a5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:55.623865 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-956c7c31-d429-4efc-9b1d-83046c17a5b8 None None] MFA Rules not processed for user `3aa95b09a72c493fb5d02f407a4918d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:55.625344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-956c7c31-d429-4efc-9b1d-83046c17a5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:55.639019 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:55.666078 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:55.686847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b538ca-b7b1-44db-aaf5-661695864dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:55.687563 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b538ca-b7b1-44db-aaf5-661695864dda admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:55.687857 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b538ca-b7b1-44db-aaf5-661695864dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:55.688139 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b538ca-b7b1-44db-aaf5-661695864dda admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:55.688865 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b538ca-b7b1-44db-aaf5-661695864dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:55.691877 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b538ca-b7b1-44db-aaf5-661695864dda admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:55.720895 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1286/2570] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:55.730141 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b538ca-b7b1-44db-aaf5-661695864dda admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:55.743349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b538ca-b7b1-44db-aaf5-661695864dda admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:55.756066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db4aa4-95b7-49f5-8df2-896e5795d128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:55.757610 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db4aa4-95b7-49f5-8df2-896e5795d128 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:55.757964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db4aa4-95b7-49f5-8df2-896e5795d128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:55.758423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1285/2571] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:55.759350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db4aa4-95b7-49f5-8df2-896e5795d128 admin admin] PATH_INFO: `/v3/projects/7b42793db03342d599c91af9d0519175/users/37f2170d8bb14e2c93eb5e9a1150f9dc/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:55.770504 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c7c31-d429-4efc-9b1d-83046c17a5b8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:55.772655 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-956c7c31-d429-4efc-9b1d-83046c17a5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:55.776989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db4aa4-95b7-49f5-8df2-896e5795d128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b42793db03342d599c91af9d0519175, user_id=37f2170d8bb14e2c93eb5e9a1150f9dc, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:55.780019 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db4aa4-95b7-49f5-8df2-896e5795d128 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:55.804734 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1287/2572] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:55 2022] PUT /identity/v3/projects/7b42793db03342d599c91af9d0519175/users/37f2170d8bb14e2c93eb5e9a1150f9dc/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:55.818529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e0b39-0672-4b2b-b3f1-3080bbfd8177 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:55.818956 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e0b39-0672-4b2b-b3f1-3080bbfd8177 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:55.819335 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e0b39-0672-4b2b-b3f1-3080bbfd8177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:55.863621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2f2e0b39-0672-4b2b-b3f1-3080bbfd8177 None None] MFA Rules not processed for user `37f2170d8bb14e2c93eb5e9a1150f9dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:55.865416 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2f2e0b39-0672-4b2b-b3f1-3080bbfd8177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:55.895705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c7c31-d429-4efc-9b1d-83046c17a5b8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:55.976680 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe06416-034d-459f-a8c9-2859f45d4491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:55.977813 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe06416-034d-459f-a8c9-2859f45d4491 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:55.978103 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe06416-034d-459f-a8c9-2859f45d4491 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:55.978560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe06416-034d-459f-a8c9-2859f45d4491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:56.070093 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1288/2573] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:56.071876 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e0b39-0672-4b2b-b3f1-3080bbfd8177 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:56.123955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe06416-034d-459f-a8c9-2859f45d4491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:56.127068 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe06416-034d-459f-a8c9-2859f45d4491 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:56.168497 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1286/2574] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:50:56.192677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe06416-034d-459f-a8c9-2859f45d4491 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:56.214706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe06416-034d-459f-a8c9-2859f45d4491 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:56.221789 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e0b39-0672-4b2b-b3f1-3080bbfd8177 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:56.237146 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf3712-3b89-484a-b61c-9f1dd3a41ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:56.237504 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf3712-3b89-484a-b61c-9f1dd3a41ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:56.237860 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf3712-3b89-484a-b61c-9f1dd3a41ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:56.238215 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf3712-3b89-484a-b61c-9f1dd3a41ed2 admin admin] PATH_INFO: `/v3/users/770f358a86db45e5ada5c0784cfec364` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:56.241161 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf3712-3b89-484a-b61c-9f1dd3a41ed2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=770f358a86db45e5ada5c0784cfec364)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:56.242421 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb06f10-a40c-4707-935b-90cc236e5df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:56.243135 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb06f10-a40c-4707-935b-90cc236e5df6 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:56.243633 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf3712-3b89-484a-b61c-9f1dd3a41ed2 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:56.244132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb06f10-a40c-4707-935b-90cc236e5df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:56.244132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb06f10-a40c-4707-935b-90cc236e5df6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:56.244923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb06f10-a40c-4707-935b-90cc236e5df6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:56.246882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb06f10-a40c-4707-935b-90cc236e5df6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:56.281445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1289/2575] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:56 2022] POST /identity/v3/projects => generated 377 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:56.282760 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb06f10-a40c-4707-935b-90cc236e5df6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:56.283721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:56.343798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:56.367833 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1eaf3712-3b89-484a-b61c-9f1dd3a41ed2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770f358a86db45e5ada5c0784cfec364'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:56.371669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb06f10-a40c-4707-935b-90cc236e5df6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:56.377740 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1eaf3712-3b89-484a-b61c-9f1dd3a41ed2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770f358a86db45e5ada5c0784cfec364'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:56.378053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f0412-04c4-40bd-b339-2670ec4ea4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:56.378763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f0412-04c4-40bd-b339-2670ec4ea4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:56.379112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f0412-04c4-40bd-b339-2670ec4ea4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:56.379452 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f0412-04c4-40bd-b339-2670ec4ea4e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:56.380439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f0412-04c4-40bd-b339-2670ec4ea4e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:56.382632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f0412-04c4-40bd-b339-2670ec4ea4e2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:56.393035 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1eaf3712-3b89-484a-b61c-9f1dd3a41ed2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770f358a86db45e5ada5c0784cfec364'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:56.395052 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1eaf3712-3b89-484a-b61c-9f1dd3a41ed2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770f358a86db45e5ada5c0784cfec364'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:56.396940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1eaf3712-3b89-484a-b61c-9f1dd3a41ed2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770f358a86db45e5ada5c0784cfec364'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:56.418418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1290/2576] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:56 2022] POST /identity/v3/users => generated 295 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:56.419680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f0412-04c4-40bd-b339-2670ec4ea4e2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:56.421768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c38f0412-04c4-40bd-b339-2670ec4ea4e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:56.430522 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1287/2577] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:56 2022] DELETE /identity/v3/users/770f358a86db45e5ada5c0784cfec364 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:56.435626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61512efd-81cc-4ba7-9496-a482e96b6e75 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:56.435626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61512efd-81cc-4ba7-9496-a482e96b6e75 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:56.435626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61512efd-81cc-4ba7-9496-a482e96b6e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:56.453998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f0412-04c4-40bd-b339-2670ec4ea4e2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:56.480995 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-61512efd-81cc-4ba7-9496-a482e96b6e75 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:56.482522 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-61512efd-81cc-4ba7-9496-a482e96b6e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:56.587290 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1288/2578] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:56.600746 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61512efd-81cc-4ba7-9496-a482e96b6e75 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:56.601772 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfc5b6-d24a-47c5-b257-534b6a471194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:56.602466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfc5b6-d24a-47c5-b257-534b6a471194 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:56.602749 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfc5b6-d24a-47c5-b257-534b6a471194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:56.602926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-61512efd-81cc-4ba7-9496-a482e96b6e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:56.603055 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfc5b6-d24a-47c5-b257-534b6a471194 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:56.604170 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfc5b6-d24a-47c5-b257-534b6a471194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:56.605887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfc5b6-d24a-47c5-b257-534b6a471194 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:56.628251 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1291/2579] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:56.635843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfc5b6-d24a-47c5-b257-534b6a471194 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:56.685015 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61512efd-81cc-4ba7-9496-a482e96b6e75 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:56.689794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfc5b6-d24a-47c5-b257-534b6a471194 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:56.693367 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d7ac7-1352-4941-9162-657323493b8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:56.694069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d7ac7-1352-4941-9162-657323493b8d service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:56.694350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d7ac7-1352-4941-9162-657323493b8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:56.694623 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d7ac7-1352-4941-9162-657323493b8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:56.709797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273facc-707a-46a9-8882-46e02670f901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:56.710882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273facc-707a-46a9-8882-46e02670f901 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:56.711488 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273facc-707a-46a9-8882-46e02670f901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:56.711986 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273facc-707a-46a9-8882-46e02670f901 admin admin] PATH_INFO: `/v3/projects/1170c2696d8c42638dc77928bcaa5164/users/edaa26ed56374210b7fd88d114cb3898/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:56.713941 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d7ac7-1352-4941-9162-657323493b8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:56.715784 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d7ac7-1352-4941-9162-657323493b8d service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:56.745729 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1289/2580] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:56.760575 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273facc-707a-46a9-8882-46e02670f901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1170c2696d8c42638dc77928bcaa5164, user_id=edaa26ed56374210b7fd88d114cb3898, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:56.766298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273facc-707a-46a9-8882-46e02670f901 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:56.790142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1292/2581] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:56 2022] PUT /identity/v3/projects/1170c2696d8c42638dc77928bcaa5164/users/edaa26ed56374210b7fd88d114cb3898/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:56.794645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d7ac7-1352-4941-9162-657323493b8d service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:56.808791 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d7ac7-1352-4941-9162-657323493b8d service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:56.892193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:56.954702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a6a63-2be5-4de1-917b-60c101970238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:56.955365 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a6a63-2be5-4de1-917b-60c101970238 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:56.955656 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a6a63-2be5-4de1-917b-60c101970238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:56.956431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a6a63-2be5-4de1-917b-60c101970238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:56.957402 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a6a63-2be5-4de1-917b-60c101970238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:56.959113 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a6a63-2be5-4de1-917b-60c101970238 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:56.976560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273facc-707a-46a9-8882-46e02670f901 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:56.982982 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1290/2582] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:56.991111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273facc-707a-46a9-8882-46e02670f901 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:57.001022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4618237-544f-4ded-9c51-331bc87eaf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:57.001738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4618237-544f-4ded-9c51-331bc87eaf15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:57.002030 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4618237-544f-4ded-9c51-331bc87eaf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:57.002312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4618237-544f-4ded-9c51-331bc87eaf15 admin admin] PATH_INFO: `/v3/projects/1170c2696d8c42638dc77928bcaa5164/users/edaa26ed56374210b7fd88d114cb3898/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:57.010909 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4618237-544f-4ded-9c51-331bc87eaf15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1170c2696d8c42638dc77928bcaa5164, user_id=edaa26ed56374210b7fd88d114cb3898, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:57.014927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4618237-544f-4ded-9c51-331bc87eaf15 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:57.059499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1293/2583] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:56 2022] PUT /identity/v3/projects/1170c2696d8c42638dc77928bcaa5164/users/edaa26ed56374210b7fd88d114cb3898/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:57.061082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d60ff-66af-4e08-99fb-f4eea61703d1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:57.061383 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d60ff-66af-4e08-99fb-f4eea61703d1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:57.061697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d60ff-66af-4e08-99fb-f4eea61703d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:57.117728 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-756d60ff-66af-4e08-99fb-f4eea61703d1 None None] MFA Rules not processed for user `edaa26ed56374210b7fd88d114cb3898`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:57.119349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-756d60ff-66af-4e08-99fb-f4eea61703d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:57.274700 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4618237-544f-4ded-9c51-331bc87eaf15 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:57.280340 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b4618237-544f-4ded-9c51-331bc87eaf15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:57.316177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1291/2584] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:50:57.334834 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d60ff-66af-4e08-99fb-f4eea61703d1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:57.338348 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-756d60ff-66af-4e08-99fb-f4eea61703d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:57.442437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4618237-544f-4ded-9c51-331bc87eaf15 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:57.447655 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d60ff-66af-4e08-99fb-f4eea61703d1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:57.448161 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53480aa2-424e-4cf3-b6cf-62da7bcca636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:57.448846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53480aa2-424e-4cf3-b6cf-62da7bcca636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:57.449134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53480aa2-424e-4cf3-b6cf-62da7bcca636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:57.449417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53480aa2-424e-4cf3-b6cf-62da7bcca636 admin admin] PATH_INFO: `/v3/projects/eca6fff00cf94e4c929295e58e4a6937` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:57.450699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53480aa2-424e-4cf3-b6cf-62da7bcca636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eca6fff00cf94e4c929295e58e4a6937)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:57.453166 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53480aa2-424e-4cf3-b6cf-62da7bcca636 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:57.457000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434f333-e588-4bdb-bde3-283970ec5d24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:57.459311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434f333-e588-4bdb-bde3-283970ec5d24 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:57.459799 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434f333-e588-4bdb-bde3-283970ec5d24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:57.464916 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434f333-e588-4bdb-bde3-283970ec5d24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:57.487613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434f333-e588-4bdb-bde3-283970ec5d24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:57.489092 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434f333-e588-4bdb-bde3-283970ec5d24 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:57.506331 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1292/2585] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:50:57.520277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:57.544959 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-53480aa2-424e-4cf3-b6cf-62da7bcca636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eca6fff00cf94e4c929295e58e4a6937', 'request_id': 'req-53480aa2-424e-4cf3-b6cf-62da7bcca636'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:57.558158 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-53480aa2-424e-4cf3-b6cf-62da7bcca636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eca6fff00cf94e4c929295e58e4a6937', 'request_id': 'req-53480aa2-424e-4cf3-b6cf-62da7bcca636'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:57.568724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-53480aa2-424e-4cf3-b6cf-62da7bcca636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eca6fff00cf94e4c929295e58e4a6937', 'request_id': 'req-53480aa2-424e-4cf3-b6cf-62da7bcca636'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:57.588798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-53480aa2-424e-4cf3-b6cf-62da7bcca636 admin admin] The token cache is being invalidate because project eca6fff00cf94e4c929295e58e4a6937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:57.589126 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-53480aa2-424e-4cf3-b6cf-62da7bcca636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:57.590702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-53480aa2-424e-4cf3-b6cf-62da7bcca636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:57.596643 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1294/2586] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:57 2022] DELETE /identity/v3/projects/eca6fff00cf94e4c929295e58e4a6937 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:57.596849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434f333-e588-4bdb-bde3-283970ec5d24 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:57.596849 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9434f333-e588-4bdb-bde3-283970ec5d24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:57.606391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53480aa2-424e-4cf3-b6cf-62da7bcca636 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:57.622901 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-53480aa2-424e-4cf3-b6cf-62da7bcca636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:57.625209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434f333-e588-4bdb-bde3-283970ec5d24 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:57.737240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b03fb-c5dd-4649-abdb-0d1342992f1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:57.738697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b03fb-c5dd-4649-abdb-0d1342992f1b service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:57.738961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b03fb-c5dd-4649-abdb-0d1342992f1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:57.739230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b03fb-c5dd-4649-abdb-0d1342992f1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:57.748944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-024b03fb-c5dd-4649-abdb-0d1342992f1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:57.759599 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53480aa2-424e-4cf3-b6cf-62da7bcca636 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:57.770805 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:57.773868 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bda70-aaa7-487b-a057-2feceb85a0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:57.774561 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bda70-aaa7-487b-a057-2feceb85a0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:57.774846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bda70-aaa7-487b-a057-2feceb85a0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:57.775117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bda70-aaa7-487b-a057-2feceb85a0b0 admin admin] PATH_INFO: `/v3/users/3aa95b09a72c493fb5d02f407a4918d5` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:57.776423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bda70-aaa7-487b-a057-2feceb85a0b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aa95b09a72c493fb5d02f407a4918d5)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:57.778509 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bda70-aaa7-487b-a057-2feceb85a0b0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:57.878398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-847bda70-aaa7-487b-a057-2feceb85a0b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aa95b09a72c493fb5d02f407a4918d5'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:57.890221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b03fb-c5dd-4649-abdb-0d1342992f1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:57.892907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b03fb-c5dd-4649-abdb-0d1342992f1b service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:57.905756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-847bda70-aaa7-487b-a057-2feceb85a0b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aa95b09a72c493fb5d02f407a4918d5'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:57.907153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-847bda70-aaa7-487b-a057-2feceb85a0b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aa95b09a72c493fb5d02f407a4918d5'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:57.909088 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1293/2587] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:50:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:50:57.929773 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-847bda70-aaa7-487b-a057-2feceb85a0b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aa95b09a72c493fb5d02f407a4918d5'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:57.953046 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-847bda70-aaa7-487b-a057-2feceb85a0b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aa95b09a72c493fb5d02f407a4918d5'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:57.968869 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1295/2588] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:50:57 2022] DELETE /identity/v3/users/3aa95b09a72c493fb5d02f407a4918d5 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:58.475425 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b03fb-c5dd-4649-abdb-0d1342992f1b service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:58.493159 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b03fb-c5dd-4649-abdb-0d1342992f1b service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:58.602796 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2695e-1684-4ed4-a93a-beae4c9bb719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:58.603578 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2695e-1684-4ed4-a93a-beae4c9bb719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:58.603932 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2695e-1684-4ed4-a93a-beae4c9bb719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:58.604391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2695e-1684-4ed4-a93a-beae4c9bb719 admin admin] PATH_INFO: `/v3/projects/7dc96ce52b734b0ca017ba7792cc1723` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:58.605829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2695e-1684-4ed4-a93a-beae4c9bb719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dc96ce52b734b0ca017ba7792cc1723)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:58.608049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2695e-1684-4ed4-a93a-beae4c9bb719 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:58.671970 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-50b2695e-1684-4ed4-a93a-beae4c9bb719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dc96ce52b734b0ca017ba7792cc1723', 'request_id': 'req-50b2695e-1684-4ed4-a93a-beae4c9bb719'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:58.678860 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-50b2695e-1684-4ed4-a93a-beae4c9bb719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dc96ce52b734b0ca017ba7792cc1723', 'request_id': 'req-50b2695e-1684-4ed4-a93a-beae4c9bb719'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:58.693927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-50b2695e-1684-4ed4-a93a-beae4c9bb719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dc96ce52b734b0ca017ba7792cc1723', 'request_id': 'req-50b2695e-1684-4ed4-a93a-beae4c9bb719'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:58.719090 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-50b2695e-1684-4ed4-a93a-beae4c9bb719 admin admin] The token cache is being invalidate because project 7dc96ce52b734b0ca017ba7792cc1723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:50:58.719426 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-50b2695e-1684-4ed4-a93a-beae4c9bb719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:58.720041 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-50b2695e-1684-4ed4-a93a-beae4c9bb719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:50:58.724782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1294/2589] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:50:58 2022] DELETE /identity/v3/projects/7dc96ce52b734b0ca017ba7792cc1723 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:58.726517 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7e88e-a17c-4ec4-b3de-f51028713d94 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:58.727026 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7e88e-a17c-4ec4-b3de-f51028713d94 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:58.727219 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7e88e-a17c-4ec4-b3de-f51028713d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:58.755354 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-12f7e88e-a17c-4ec4-b3de-f51028713d94 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:58.757381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-12f7e88e-a17c-4ec4-b3de-f51028713d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:58.795220 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:58.833021 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1296/2590] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:58.841608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd459f-ab03-48b0-99ea-5b71e2c560f9 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:58.842305 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd459f-ab03-48b0-99ea-5b71e2c560f9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:58.842630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd459f-ab03-48b0-99ea-5b71e2c560f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:58.872289 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-37fd459f-ab03-48b0-99ea-5b71e2c560f9 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:58.873656 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-37fd459f-ab03-48b0-99ea-5b71e2c560f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:58.884906 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1295/2591] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:58.889094 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7e88e-a17c-4ec4-b3de-f51028713d94 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:58.904251 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7e88e-a17c-4ec4-b3de-f51028713d94 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:58.908989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d7c75-64fb-4e73-9e3e-16cc6c994b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:58.909752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d7c75-64fb-4e73-9e3e-16cc6c994b61 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:58.910034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d7c75-64fb-4e73-9e3e-16cc6c994b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:58.910310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d7c75-64fb-4e73-9e3e-16cc6c994b61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:58.911011 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d7c75-64fb-4e73-9e3e-16cc6c994b61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:50:58.911315 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d7c75-64fb-4e73-9e3e-16cc6c994b61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:58.912947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d7c75-64fb-4e73-9e3e-16cc6c994b61 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:58.941837 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1297/2592] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:50:58.942677 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd459f-ab03-48b0-99ea-5b71e2c560f9 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:58.959656 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd459f-ab03-48b0-99ea-5b71e2c560f9 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:58.965927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95aaaa-bf9b-4984-adbe-68cf93f15cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:58.966561 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95aaaa-bf9b-4984-adbe-68cf93f15cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:58.966838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95aaaa-bf9b-4984-adbe-68cf93f15cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:58.967110 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95aaaa-bf9b-4984-adbe-68cf93f15cf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:58.967774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95aaaa-bf9b-4984-adbe-68cf93f15cf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:58.970378 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95aaaa-bf9b-4984-adbe-68cf93f15cf1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:59.015193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1296/2593] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:58 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:59.019193 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d7c75-64fb-4e73-9e3e-16cc6c994b61 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:59.031424 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d7c75-64fb-4e73-9e3e-16cc6c994b61 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:59.113948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1f4ae-3e28-48ea-9d3d-a8d1444c03c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:59.115126 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1f4ae-3e28-48ea-9d3d-a8d1444c03c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:59.115407 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1f4ae-3e28-48ea-9d3d-a8d1444c03c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:59.115704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1f4ae-3e28-48ea-9d3d-a8d1444c03c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:59.116391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1f4ae-3e28-48ea-9d3d-a8d1444c03c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:59.119165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1f4ae-3e28-48ea-9d3d-a8d1444c03c0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:59.163769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1298/2594] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:59 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:59.166468 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95aaaa-bf9b-4984-adbe-68cf93f15cf1 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:59.195074 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95aaaa-bf9b-4984-adbe-68cf93f15cf1 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:59.208141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84570a0-eecd-43b0-b4e1-572bcbeaf0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:59.209558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84570a0-eecd-43b0-b4e1-572bcbeaf0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:59.210162 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84570a0-eecd-43b0-b4e1-572bcbeaf0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:59.210692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84570a0-eecd-43b0-b4e1-572bcbeaf0b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:59.217167 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84570a0-eecd-43b0-b4e1-572bcbeaf0b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:59.220670 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84570a0-eecd-43b0-b4e1-572bcbeaf0b6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:59.229877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:59.239267 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1297/2595] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:50:59.244447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1f4ae-3e28-48ea-9d3d-a8d1444c03c0 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:59.257802 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1f4ae-3e28-48ea-9d3d-a8d1444c03c0 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:59.264428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17977954-a142-427a-8c66-dd49a4df911e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:59.265081 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17977954-a142-427a-8c66-dd49a4df911e admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:59.265404 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17977954-a142-427a-8c66-dd49a4df911e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:59.265728 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17977954-a142-427a-8c66-dd49a4df911e admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:59.266369 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17977954-a142-427a-8c66-dd49a4df911e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:59.268122 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17977954-a142-427a-8c66-dd49a4df911e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:59.285940 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1299/2596] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:50:59.286621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84570a0-eecd-43b0-b4e1-572bcbeaf0b6 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:59.298139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84570a0-eecd-43b0-b4e1-572bcbeaf0b6 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:59.304166 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b3afa-d5e8-40c8-b9fe-480bec04bcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:59.305006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b3afa-d5e8-40c8-b9fe-480bec04bcbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:59.305350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b3afa-d5e8-40c8-b9fe-480bec04bcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:59.305684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b3afa-d5e8-40c8-b9fe-480bec04bcbd admin admin] PATH_INFO: `/v3/projects/bed9dc040e3d46eda7c0edc2a351ef19/users/c2835956436b436facaee3ce2be86759/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:59.329316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b3afa-d5e8-40c8-b9fe-480bec04bcbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed9dc040e3d46eda7c0edc2a351ef19, user_id=c2835956436b436facaee3ce2be86759, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:59.332077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b3afa-d5e8-40c8-b9fe-480bec04bcbd admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:59.349337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1298/2597] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:59 2022] PUT /identity/v3/projects/bed9dc040e3d46eda7c0edc2a351ef19/users/c2835956436b436facaee3ce2be86759/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:50:59.363369 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6a176-1c0f-4197-8fa3-285b9d90c64d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:59.363793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6a176-1c0f-4197-8fa3-285b9d90c64d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:59.364198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6a176-1c0f-4197-8fa3-285b9d90c64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:59.395867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5cd6a176-1c0f-4197-8fa3-285b9d90c64d None None] MFA Rules not processed for user `c2835956436b436facaee3ce2be86759`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:50:59.397247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5cd6a176-1c0f-4197-8fa3-285b9d90c64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:50:59.474990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1300/2598] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:50:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:50:59.479687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b3afa-d5e8-40c8-b9fe-480bec04bcbd admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:59.551335 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b3afa-d5e8-40c8-b9fe-480bec04bcbd admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:59.556554 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2fe55b-2998-4fdc-9636-8b8346ebc17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:59.557515 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fe55b-2998-4fdc-9636-8b8346ebc17e admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:59.558655 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fe55b-2998-4fdc-9636-8b8346ebc17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:59.559887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fe55b-2998-4fdc-9636-8b8346ebc17e admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:59.560584 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2fe55b-2998-4fdc-9636-8b8346ebc17e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:59.562628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2fe55b-2998-4fdc-9636-8b8346ebc17e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:59.598225 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1299/2599] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:50:59 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:50:59.601879 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6a176-1c0f-4197-8fa3-285b9d90c64d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:59.615851 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6a176-1c0f-4197-8fa3-285b9d90c64d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:59.660824 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:50:59.705364 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8baafc-39ae-481e-ae8c-d6db3da42570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:59.708037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8baafc-39ae-481e-ae8c-d6db3da42570 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:59.708328 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8baafc-39ae-481e-ae8c-d6db3da42570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:59.708607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8baafc-39ae-481e-ae8c-d6db3da42570 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:59.709333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8baafc-39ae-481e-ae8c-d6db3da42570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:59.711318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8baafc-39ae-481e-ae8c-d6db3da42570 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:59.764433 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1301/2600] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:50:59 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:50:59.766430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2fe55b-2998-4fdc-9636-8b8346ebc17e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:59.782031 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2fe55b-2998-4fdc-9636-8b8346ebc17e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:59.788630 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acfc2d-67ce-458d-a0c7-f4cc05f80957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:59.789700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acfc2d-67ce-458d-a0c7-f4cc05f80957 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:59.790221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acfc2d-67ce-458d-a0c7-f4cc05f80957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:59.790736 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acfc2d-67ce-458d-a0c7-f4cc05f80957 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:59.791904 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87acfc2d-67ce-458d-a0c7-f4cc05f80957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:59.793813 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87acfc2d-67ce-458d-a0c7-f4cc05f80957 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:59.806944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1300/2601] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:50:59.811184 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8baafc-39ae-481e-ae8c-d6db3da42570 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:59.826523 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8baafc-39ae-481e-ae8c-d6db3da42570 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:59.832022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b5b8a-c634-4ff6-b66c-513367104fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:59.832914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b5b8a-c634-4ff6-b66c-513367104fb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:59.833303 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b5b8a-c634-4ff6-b66c-513367104fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:59.833764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b5b8a-c634-4ff6-b66c-513367104fb3 admin admin] PATH_INFO: `/v3/projects/5780dda951ae4c94acdac0ffd6b20429/users/bf16641a027a472c85322da6e11d98a8/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:59.859943 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b5b8a-c634-4ff6-b66c-513367104fb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5780dda951ae4c94acdac0ffd6b20429, user_id=bf16641a027a472c85322da6e11d98a8, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:59.863039 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b5b8a-c634-4ff6-b66c-513367104fb3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:50:59.883973 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1302/2602] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:50:59 2022] PUT /identity/v3/projects/5780dda951ae4c94acdac0ffd6b20429/users/bf16641a027a472c85322da6e11d98a8/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:50:59.887802 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acfc2d-67ce-458d-a0c7-f4cc05f80957 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:50:59.901850 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acfc2d-67ce-458d-a0c7-f4cc05f80957 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:50:59.982512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1149588-9484-4764-8628-5d2a942e9948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:50:59.983649 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1149588-9484-4764-8628-5d2a942e9948 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:50:59.984133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1149588-9484-4764-8628-5d2a942e9948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:50:59.984585 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1149588-9484-4764-8628-5d2a942e9948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:50:59.985327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1149588-9484-4764-8628-5d2a942e9948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:50:59.987087 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1149588-9484-4764-8628-5d2a942e9948 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:00.008353 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1301/2603] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:51:00.013065 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b5b8a-c634-4ff6-b66c-513367104fb3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:00.026050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b5b8a-c634-4ff6-b66c-513367104fb3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:00.032339 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bab58f-6344-4fce-85ff-5582bdadf31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:00.032991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bab58f-6344-4fce-85ff-5582bdadf31b admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:00.033347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bab58f-6344-4fce-85ff-5582bdadf31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:00.033730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bab58f-6344-4fce-85ff-5582bdadf31b admin admin] PATH_INFO: `/v3/projects/5780dda951ae4c94acdac0ffd6b20429/users/bf16641a027a472c85322da6e11d98a8/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:00.036377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bab58f-6344-4fce-85ff-5582bdadf31b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5780dda951ae4c94acdac0ffd6b20429, user_id=bf16641a027a472c85322da6e11d98a8, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:00.043482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bab58f-6344-4fce-85ff-5582bdadf31b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:00.058276 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1303/2604] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:51:00 2022] PUT /identity/v3/projects/5780dda951ae4c94acdac0ffd6b20429/users/bf16641a027a472c85322da6e11d98a8/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:00.072479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045eeb1d-77e0-4699-a6aa-d974c41f02c5 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:00.073025 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045eeb1d-77e0-4699-a6aa-d974c41f02c5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:00.073323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045eeb1d-77e0-4699-a6aa-d974c41f02c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:00.125510 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-045eeb1d-77e0-4699-a6aa-d974c41f02c5 None None] MFA Rules not processed for user `bf16641a027a472c85322da6e11d98a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:00.125510 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-045eeb1d-77e0-4699-a6aa-d974c41f02c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:00.177890 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:00.219743 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1302/2605] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:00.232203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bab58f-6344-4fce-85ff-5582bdadf31b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:00.234059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-33bab58f-6344-4fce-85ff-5582bdadf31b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:00.332595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bab58f-6344-4fce-85ff-5582bdadf31b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:00.337421 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593ba06-1c10-4f28-8b28-8b2590402295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:00.338347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593ba06-1c10-4f28-8b28-8b2590402295 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:00.338841 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593ba06-1c10-4f28-8b28-8b2590402295 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:00.339123 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593ba06-1c10-4f28-8b28-8b2590402295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:00.356406 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593ba06-1c10-4f28-8b28-8b2590402295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:00.360634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593ba06-1c10-4f28-8b28-8b2590402295 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:00.378864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1304/2606] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:01.181037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045eeb1d-77e0-4699-a6aa-d974c41f02c5 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:01.313193 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593ba06-1c10-4f28-8b28-8b2590402295 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:01.314800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e593ba06-1c10-4f28-8b28-8b2590402295 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:01.318430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045eeb1d-77e0-4699-a6aa-d974c41f02c5 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:01.324813 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce216d9-3dbd-4c39-80d7-7f59bc99968f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:01.325850 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce216d9-3dbd-4c39-80d7-7f59bc99968f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:01.326297 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce216d9-3dbd-4c39-80d7-7f59bc99968f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:01.326786 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce216d9-3dbd-4c39-80d7-7f59bc99968f admin admin] PATH_INFO: `/v3/users/c2835956436b436facaee3ce2be86759` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:01.328291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce216d9-3dbd-4c39-80d7-7f59bc99968f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2835956436b436facaee3ce2be86759)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:01.330668 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593ba06-1c10-4f28-8b28-8b2590402295 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:01.331636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce216d9-3dbd-4c39-80d7-7f59bc99968f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:01.339953 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafecfe2-47cd-43a2-935c-eb27b308e5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:01.340779 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafecfe2-47cd-43a2-935c-eb27b308e5f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:01.341204 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafecfe2-47cd-43a2-935c-eb27b308e5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:01.341583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafecfe2-47cd-43a2-935c-eb27b308e5f4 admin admin] PATH_INFO: `/v3/users/1c7f3fb6979b437bbaa8f80930ca6f06` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:01.350336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafecfe2-47cd-43a2-935c-eb27b308e5f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c7f3fb6979b437bbaa8f80930ca6f06)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:01.356043 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafecfe2-47cd-43a2-935c-eb27b308e5f4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:01.426349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-0ce216d9-3dbd-4c39-80d7-7f59bc99968f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2835956436b436facaee3ce2be86759'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:01.435823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-dafecfe2-47cd-43a2-935c-eb27b308e5f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c7f3fb6979b437bbaa8f80930ca6f06'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:01.437298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-0ce216d9-3dbd-4c39-80d7-7f59bc99968f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2835956436b436facaee3ce2be86759'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:01.452846 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-0ce216d9-3dbd-4c39-80d7-7f59bc99968f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2835956436b436facaee3ce2be86759'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:01.455502 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-0ce216d9-3dbd-4c39-80d7-7f59bc99968f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2835956436b436facaee3ce2be86759'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:01.457026 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-0ce216d9-3dbd-4c39-80d7-7f59bc99968f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2835956436b436facaee3ce2be86759'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:01.473061 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-dafecfe2-47cd-43a2-935c-eb27b308e5f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c7f3fb6979b437bbaa8f80930ca6f06'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:01.476025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-dafecfe2-47cd-43a2-935c-eb27b308e5f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c7f3fb6979b437bbaa8f80930ca6f06'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:01.493807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1303/2607] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:01 2022] DELETE /identity/v3/users/c2835956436b436facaee3ce2be86759 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:01.495866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-dafecfe2-47cd-43a2-935c-eb27b308e5f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c7f3fb6979b437bbaa8f80930ca6f06'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:01.496399 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae53ac6-8e68-4068-804e-b6759968da85 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:01.496758 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae53ac6-8e68-4068-804e-b6759968da85 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:01.497048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae53ac6-8e68-4068-804e-b6759968da85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:01.511015 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-dafecfe2-47cd-43a2-935c-eb27b308e5f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c7f3fb6979b437bbaa8f80930ca6f06'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:01.513176 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:01.529604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1305/2608] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:01 2022] DELETE /identity/v3/users/1c7f3fb6979b437bbaa8f80930ca6f06 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:01.532063 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-9ae53ac6-8e68-4068-804e-b6759968da85 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:01.533918 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9ae53ac6-8e68-4068-804e-b6759968da85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:01.540415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0f098-8dad-49c5-a42a-04bd96d69e2d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:01.542564 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0f098-8dad-49c5-a42a-04bd96d69e2d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:01.546009 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0f098-8dad-49c5-a42a-04bd96d69e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:01.585668 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-41d0f098-8dad-49c5-a42a-04bd96d69e2d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:01.587574 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-41d0f098-8dad-49c5-a42a-04bd96d69e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:01.628146 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:01.651164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1304/2609] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:01.663370 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae53ac6-8e68-4068-804e-b6759968da85 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:01.666676 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9ae53ac6-8e68-4068-804e-b6759968da85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:01.738643 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1306/2610] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:01.751588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0f098-8dad-49c5-a42a-04bd96d69e2d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:01.778466 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0f098-8dad-49c5-a42a-04bd96d69e2d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:01.779020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae53ac6-8e68-4068-804e-b6759968da85 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:01.783672 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad660970-5f08-4538-b12b-aa61dd53ad6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:01.784524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c915c-2abb-46f3-9c26-8124878ef81e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:01.784721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad660970-5f08-4538-b12b-aa61dd53ad6d service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:01.785192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad660970-5f08-4538-b12b-aa61dd53ad6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:01.785305 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c915c-2abb-46f3-9c26-8124878ef81e service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:01.785660 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c915c-2abb-46f3-9c26-8124878ef81e service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:01.785792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad660970-5f08-4538-b12b-aa61dd53ad6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:01.785991 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c915c-2abb-46f3-9c26-8124878ef81e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:01.807089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c915c-2abb-46f3-9c26-8124878ef81e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:01.808640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c915c-2abb-46f3-9c26-8124878ef81e service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:01.828197 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad660970-5f08-4538-b12b-aa61dd53ad6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:01.830881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1305/2611] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:51:01.831196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad660970-5f08-4538-b12b-aa61dd53ad6d service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:01.844471 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1307/2612] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:02.371028 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c915c-2abb-46f3-9c26-8124878ef81e service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:02.374698 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cc4c915c-2abb-46f3-9c26-8124878ef81e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:02.400111 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad660970-5f08-4538-b12b-aa61dd53ad6d service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:02.401776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ad660970-5f08-4538-b12b-aa61dd53ad6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:02.411052 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c915c-2abb-46f3-9c26-8124878ef81e service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:02.426079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85f898-0116-4e5f-92f1-f6524fadf09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:02.428543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85f898-0116-4e5f-92f1-f6524fadf09e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:02.431852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85f898-0116-4e5f-92f1-f6524fadf09e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:02.432719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85f898-0116-4e5f-92f1-f6524fadf09e admin admin] PATH_INFO: `/v3/projects/0c9f244aebdf4d3e9a379b0bb999fa56` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:02.434656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad660970-5f08-4538-b12b-aa61dd53ad6d service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:02.436108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85f898-0116-4e5f-92f1-f6524fadf09e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c9f244aebdf4d3e9a379b0bb999fa56)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:02.443307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85f898-0116-4e5f-92f1-f6524fadf09e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:02.444500 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7841d7b-0e0c-477c-87bd-37b16ce9a670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:02.446325 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7841d7b-0e0c-477c-87bd-37b16ce9a670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:02.446630 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7841d7b-0e0c-477c-87bd-37b16ce9a670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:02.446913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7841d7b-0e0c-477c-87bd-37b16ce9a670 admin admin] PATH_INFO: `/v3/projects/bed9dc040e3d46eda7c0edc2a351ef19` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:02.448256 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7841d7b-0e0c-477c-87bd-37b16ce9a670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bed9dc040e3d46eda7c0edc2a351ef19)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:02.451495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7841d7b-0e0c-477c-87bd-37b16ce9a670 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:02.519567 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c7841d7b-0e0c-477c-87bd-37b16ce9a670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bed9dc040e3d46eda7c0edc2a351ef19', 'request_id': 'req-c7841d7b-0e0c-477c-87bd-37b16ce9a670'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.531888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c7841d7b-0e0c-477c-87bd-37b16ce9a670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bed9dc040e3d46eda7c0edc2a351ef19', 'request_id': 'req-c7841d7b-0e0c-477c-87bd-37b16ce9a670'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.541810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c7841d7b-0e0c-477c-87bd-37b16ce9a670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bed9dc040e3d46eda7c0edc2a351ef19', 'request_id': 'req-c7841d7b-0e0c-477c-87bd-37b16ce9a670'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.553093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8e85f898-0116-4e5f-92f1-f6524fadf09e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c9f244aebdf4d3e9a379b0bb999fa56', 'request_id': 'req-8e85f898-0116-4e5f-92f1-f6524fadf09e'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.562343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8e85f898-0116-4e5f-92f1-f6524fadf09e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c9f244aebdf4d3e9a379b0bb999fa56', 'request_id': 'req-8e85f898-0116-4e5f-92f1-f6524fadf09e'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.564705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c7841d7b-0e0c-477c-87bd-37b16ce9a670 admin admin] The token cache is being invalidate because project bed9dc040e3d46eda7c0edc2a351ef19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:51:02.565377 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c7841d7b-0e0c-477c-87bd-37b16ce9a670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.567388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c7841d7b-0e0c-477c-87bd-37b16ce9a670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.569612 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1308/2613] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:51:02 2022] DELETE /identity/v3/projects/bed9dc040e3d46eda7c0edc2a351ef19 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:02.573842 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7841d7b-0e0c-477c-87bd-37b16ce9a670 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:02.576662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c7841d7b-0e0c-477c-87bd-37b16ce9a670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:02.578391 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8e85f898-0116-4e5f-92f1-f6524fadf09e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c9f244aebdf4d3e9a379b0bb999fa56', 'request_id': 'req-8e85f898-0116-4e5f-92f1-f6524fadf09e'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.603574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8e85f898-0116-4e5f-92f1-f6524fadf09e admin admin] The token cache is being invalidate because project 0c9f244aebdf4d3e9a379b0bb999fa56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:51:02.604133 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8e85f898-0116-4e5f-92f1-f6524fadf09e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.605250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8e85f898-0116-4e5f-92f1-f6524fadf09e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.610357 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1306/2614] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:51:02 2022] DELETE /identity/v3/projects/0c9f244aebdf4d3e9a379b0bb999fa56 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:02.611556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85f898-0116-4e5f-92f1-f6524fadf09e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:02.613315 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8e85f898-0116-4e5f-92f1-f6524fadf09e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:02.712340 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85f898-0116-4e5f-92f1-f6524fadf09e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:02.718010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797bebc-eb87-41aa-b809-bc0b83dd9a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:02.718739 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797bebc-eb87-41aa-b809-bc0b83dd9a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:02.719122 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797bebc-eb87-41aa-b809-bc0b83dd9a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:02.719303 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797bebc-eb87-41aa-b809-bc0b83dd9a93 admin admin] PATH_INFO: `/v3/users/37f2170d8bb14e2c93eb5e9a1150f9dc` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:02.720598 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797bebc-eb87-41aa-b809-bc0b83dd9a93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37f2170d8bb14e2c93eb5e9a1150f9dc)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:02.723457 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7841d7b-0e0c-477c-87bd-37b16ce9a670 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:02.724928 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797bebc-eb87-41aa-b809-bc0b83dd9a93 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:02.735167 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19c6aa-01e6-4ef6-a7f9-c5bc835abaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:02.738304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19c6aa-01e6-4ef6-a7f9-c5bc835abaca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:02.738304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19c6aa-01e6-4ef6-a7f9-c5bc835abaca admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:02.738304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19c6aa-01e6-4ef6-a7f9-c5bc835abaca admin admin] PATH_INFO: `/v3/users/bf16641a027a472c85322da6e11d98a8` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:02.738304 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19c6aa-01e6-4ef6-a7f9-c5bc835abaca admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf16641a027a472c85322da6e11d98a8)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:02.740268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19c6aa-01e6-4ef6-a7f9-c5bc835abaca admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:02.756020 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:02.770163 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:02.823039 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e797bebc-eb87-41aa-b809-bc0b83dd9a93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37f2170d8bb14e2c93eb5e9a1150f9dc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.837665 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e797bebc-eb87-41aa-b809-bc0b83dd9a93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37f2170d8bb14e2c93eb5e9a1150f9dc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.840341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3a19c6aa-01e6-4ef6-a7f9-c5bc835abaca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf16641a027a472c85322da6e11d98a8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.862373 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e797bebc-eb87-41aa-b809-bc0b83dd9a93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37f2170d8bb14e2c93eb5e9a1150f9dc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.863351 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e797bebc-eb87-41aa-b809-bc0b83dd9a93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37f2170d8bb14e2c93eb5e9a1150f9dc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.865242 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e797bebc-eb87-41aa-b809-bc0b83dd9a93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37f2170d8bb14e2c93eb5e9a1150f9dc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.877563 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3a19c6aa-01e6-4ef6-a7f9-c5bc835abaca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf16641a027a472c85322da6e11d98a8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.877563 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3a19c6aa-01e6-4ef6-a7f9-c5bc835abaca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf16641a027a472c85322da6e11d98a8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.888188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3a19c6aa-01e6-4ef6-a7f9-c5bc835abaca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf16641a027a472c85322da6e11d98a8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.899362 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3a19c6aa-01e6-4ef6-a7f9-c5bc835abaca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf16641a027a472c85322da6e11d98a8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:02.907334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1307/2615] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:02 2022] DELETE /identity/v3/users/37f2170d8bb14e2c93eb5e9a1150f9dc => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:02.909528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1309/2616] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:02 2022] DELETE /identity/v3/users/bf16641a027a472c85322da6e11d98a8 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:03.460111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797bebc-eb87-41aa-b809-bc0b83dd9a93 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:03.462845 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e797bebc-eb87-41aa-b809-bc0b83dd9a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:03.587661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19c6aa-01e6-4ef6-a7f9-c5bc835abaca admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:03.593519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3a19c6aa-01e6-4ef6-a7f9-c5bc835abaca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:03.639591 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797bebc-eb87-41aa-b809-bc0b83dd9a93 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:03.724766 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19c6aa-01e6-4ef6-a7f9-c5bc835abaca admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:03.732369 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133ff45-8dfc-4938-993d-3731b780face admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:03.734012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133ff45-8dfc-4938-993d-3731b780face admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:03.734326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133ff45-8dfc-4938-993d-3731b780face admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:03.734622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133ff45-8dfc-4938-993d-3731b780face admin admin] PATH_INFO: `/v3/projects/5780dda951ae4c94acdac0ffd6b20429` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:03.736142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133ff45-8dfc-4938-993d-3731b780face admin admin] RBAC: Authorizing `identity:delete_project(project_id=5780dda951ae4c94acdac0ffd6b20429)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:03.738347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133ff45-8dfc-4938-993d-3731b780face admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:03.759162 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d14346-eabc-4b53-b9be-99162ad4069f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:03.760386 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d14346-eabc-4b53-b9be-99162ad4069f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:03.760893 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d14346-eabc-4b53-b9be-99162ad4069f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:03.761490 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d14346-eabc-4b53-b9be-99162ad4069f admin admin] PATH_INFO: `/v3/projects/7b42793db03342d599c91af9d0519175` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:03.764076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d14346-eabc-4b53-b9be-99162ad4069f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b42793db03342d599c91af9d0519175)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:03.772826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d14346-eabc-4b53-b9be-99162ad4069f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:03.821880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d133ff45-8dfc-4938-993d-3731b780face admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5780dda951ae4c94acdac0ffd6b20429', 'request_id': 'req-d133ff45-8dfc-4938-993d-3731b780face'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:03.837695 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d133ff45-8dfc-4938-993d-3731b780face admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5780dda951ae4c94acdac0ffd6b20429', 'request_id': 'req-d133ff45-8dfc-4938-993d-3731b780face'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:03.853018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d133ff45-8dfc-4938-993d-3731b780face admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5780dda951ae4c94acdac0ffd6b20429', 'request_id': 'req-d133ff45-8dfc-4938-993d-3731b780face'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:03.869442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:03.881835 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d133ff45-8dfc-4938-993d-3731b780face admin admin] The token cache is being invalidate because project 5780dda951ae4c94acdac0ffd6b20429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:51:03.882443 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d133ff45-8dfc-4938-993d-3731b780face admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:03.883681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b9d14346-eabc-4b53-b9be-99162ad4069f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b42793db03342d599c91af9d0519175', 'request_id': 'req-b9d14346-eabc-4b53-b9be-99162ad4069f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:03.883850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d133ff45-8dfc-4938-993d-3731b780face admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:03.888668 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1310/2617] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:51:03 2022] DELETE /identity/v3/projects/5780dda951ae4c94acdac0ffd6b20429 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:03.894524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b9d14346-eabc-4b53-b9be-99162ad4069f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b42793db03342d599c91af9d0519175', 'request_id': 'req-b9d14346-eabc-4b53-b9be-99162ad4069f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:03.895715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372bfd9-6f03-486c-b779-c36fa2797b55 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:03.895715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372bfd9-6f03-486c-b779-c36fa2797b55 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:03.896036 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372bfd9-6f03-486c-b779-c36fa2797b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:03.926841 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b9d14346-eabc-4b53-b9be-99162ad4069f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b42793db03342d599c91af9d0519175', 'request_id': 'req-b9d14346-eabc-4b53-b9be-99162ad4069f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:03.940451 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1372bfd9-6f03-486c-b779-c36fa2797b55 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:03.942489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1372bfd9-6f03-486c-b779-c36fa2797b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:03.960337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b9d14346-eabc-4b53-b9be-99162ad4069f admin admin] The token cache is being invalidate because project 7b42793db03342d599c91af9d0519175 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:51:03.961841 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b9d14346-eabc-4b53-b9be-99162ad4069f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:03.964249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b9d14346-eabc-4b53-b9be-99162ad4069f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:03.968304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1308/2618] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:51:03 2022] DELETE /identity/v3/projects/7b42793db03342d599c91af9d0519175 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:03.974471 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d14346-eabc-4b53-b9be-99162ad4069f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:03.975777 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:03.980746 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b9d14346-eabc-4b53-b9be-99162ad4069f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:04.083459 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1311/2619] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:04.093322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00638fe-3294-4bd7-80c2-ee6ee56480a4 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:04.097358 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00638fe-3294-4bd7-80c2-ee6ee56480a4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:04.097358 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00638fe-3294-4bd7-80c2-ee6ee56480a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:04.118545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d14346-eabc-4b53-b9be-99162ad4069f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:04.122965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c00638fe-3294-4bd7-80c2-ee6ee56480a4 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:04.124328 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c00638fe-3294-4bd7-80c2-ee6ee56480a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:04.134082 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd434c0-8f66-49fd-9ab2-57666b7392e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:04.135803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd434c0-8f66-49fd-9ab2-57666b7392e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:04.136571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd434c0-8f66-49fd-9ab2-57666b7392e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:04.137198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1312/2620] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:04.137588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd434c0-8f66-49fd-9ab2-57666b7392e2 admin admin] PATH_INFO: `/v3/users/edaa26ed56374210b7fd88d114cb3898` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:04.142846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00638fe-3294-4bd7-80c2-ee6ee56480a4 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:04.144332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd434c0-8f66-49fd-9ab2-57666b7392e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edaa26ed56374210b7fd88d114cb3898)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:04.147912 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd434c0-8f66-49fd-9ab2-57666b7392e2 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:04.155025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00638fe-3294-4bd7-80c2-ee6ee56480a4 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:04.163164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5bca4-ba7c-4a43-877e-007361e3957d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:04.163785 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5bca4-ba7c-4a43-877e-007361e3957d admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:04.164168 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5bca4-ba7c-4a43-877e-007361e3957d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:04.164591 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5bca4-ba7c-4a43-877e-007361e3957d admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:04.165428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5bca4-ba7c-4a43-877e-007361e3957d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:51:04.165866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5bca4-ba7c-4a43-877e-007361e3957d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:04.167711 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5bca4-ba7c-4a43-877e-007361e3957d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:04.185562 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1313/2621] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:51:04.191218 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5bca4-ba7c-4a43-877e-007361e3957d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:04.204532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5bca4-ba7c-4a43-877e-007361e3957d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:04.209891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724feeca-4622-4770-b18d-8c836e91e809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:04.210617 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724feeca-4622-4770-b18d-8c836e91e809 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:04.210998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724feeca-4622-4770-b18d-8c836e91e809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:04.211380 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724feeca-4622-4770-b18d-8c836e91e809 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:04.212154 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724feeca-4622-4770-b18d-8c836e91e809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:04.214644 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724feeca-4622-4770-b18d-8c836e91e809 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:04.261958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1314/2622] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:51:04 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:51:04.263962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724feeca-4622-4770-b18d-8c836e91e809 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:04.266747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-bdd434c0-8f66-49fd-9ab2-57666b7392e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edaa26ed56374210b7fd88d114cb3898'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:04.286788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-bdd434c0-8f66-49fd-9ab2-57666b7392e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edaa26ed56374210b7fd88d114cb3898'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:04.299535 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-bdd434c0-8f66-49fd-9ab2-57666b7392e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edaa26ed56374210b7fd88d114cb3898'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:04.300298 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-bdd434c0-8f66-49fd-9ab2-57666b7392e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edaa26ed56374210b7fd88d114cb3898'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:04.310696 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-bdd434c0-8f66-49fd-9ab2-57666b7392e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edaa26ed56374210b7fd88d114cb3898'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:04.352825 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1309/2623] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:03 2022] DELETE /identity/v3/users/edaa26ed56374210b7fd88d114cb3898 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:04.404908 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724feeca-4622-4770-b18d-8c836e91e809 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:04.550282 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613144a4-d109-4706-b5b4-40bf086f14f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:04.551529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613144a4-d109-4706-b5b4-40bf086f14f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:04.552188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613144a4-d109-4706-b5b4-40bf086f14f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:04.552779 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613144a4-d109-4706-b5b4-40bf086f14f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:04.554422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613144a4-d109-4706-b5b4-40bf086f14f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:04.558033 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613144a4-d109-4706-b5b4-40bf086f14f9 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:04.616229 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1315/2624] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:51:04 2022] POST /identity/v3/users => generated 288 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:51:04.619555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd434c0-8f66-49fd-9ab2-57666b7392e2 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:04.621926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bdd434c0-8f66-49fd-9ab2-57666b7392e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:04.648545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd434c0-8f66-49fd-9ab2-57666b7392e2 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:04.655542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b69772-9ac9-412d-9741-b6f259913416 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:04.656024 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b69772-9ac9-412d-9741-b6f259913416 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:04.656505 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b69772-9ac9-412d-9741-b6f259913416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:04.659773 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6983343-34cb-4ce4-8ae1-f9c1d8122feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:04.660973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6983343-34cb-4ce4-8ae1-f9c1d8122feb admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:04.661643 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6983343-34cb-4ce4-8ae1-f9c1d8122feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:04.662247 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6983343-34cb-4ce4-8ae1-f9c1d8122feb admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:04.663371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6983343-34cb-4ce4-8ae1-f9c1d8122feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:04.667319 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6983343-34cb-4ce4-8ae1-f9c1d8122feb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:04.686868 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1310/2625] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:51:04.692661 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6983343-34cb-4ce4-8ae1-f9c1d8122feb admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:04.701148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-37b69772-9ac9-412d-9741-b6f259913416 None None] MFA Rules not processed for user `538d552b04a8446094d2e84952804795`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:04.702460 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-37b69772-9ac9-412d-9741-b6f259913416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:04.713915 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:04.714212 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6983343-34cb-4ce4-8ae1-f9c1d8122feb admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:04.724308 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20596b82-3ba6-426b-9091-5791fc8b0e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:04.725529 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20596b82-3ba6-426b-9091-5791fc8b0e86 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:04.726096 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20596b82-3ba6-426b-9091-5791fc8b0e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:04.726598 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20596b82-3ba6-426b-9091-5791fc8b0e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:04.728906 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20596b82-3ba6-426b-9091-5791fc8b0e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:04.730997 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20596b82-3ba6-426b-9091-5791fc8b0e86 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:04.755465 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1311/2626] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:51:04.759201 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20596b82-3ba6-426b-9091-5791fc8b0e86 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:04.777966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1316/2627] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:04.778808 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20596b82-3ba6-426b-9091-5791fc8b0e86 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:04.786842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c59a68-32df-4849-8449-5a2eefce7e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:04.787789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c59a68-32df-4849-8449-5a2eefce7e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:04.788233 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c59a68-32df-4849-8449-5a2eefce7e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:04.788775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c59a68-32df-4849-8449-5a2eefce7e9f admin admin] PATH_INFO: `/v3/projects/805b8544ac5c4f9b96f6bb0435e30574/users/13e30ef7b13a48018e9c0974aa14483d/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:04.823895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c59a68-32df-4849-8449-5a2eefce7e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=805b8544ac5c4f9b96f6bb0435e30574, user_id=13e30ef7b13a48018e9c0974aa14483d, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:04.829218 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c59a68-32df-4849-8449-5a2eefce7e9f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:04.830996 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b69772-9ac9-412d-9741-b6f259913416 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:04.832522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-37b69772-9ac9-412d-9741-b6f259913416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:04.858662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b69772-9ac9-412d-9741-b6f259913416 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:04.859265 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1312/2628] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:51:04 2022] PUT /identity/v3/projects/805b8544ac5c4f9b96f6bb0435e30574/users/13e30ef7b13a48018e9c0974aa14483d/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:51:04.872775 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4616a09-d722-4f12-a095-355d199fd927 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:04.873494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4616a09-d722-4f12-a095-355d199fd927 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:04.874091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4616a09-d722-4f12-a095-355d199fd927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:04.900527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:04.928546 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d4616a09-d722-4f12-a095-355d199fd927 None None] MFA Rules not processed for user `13e30ef7b13a48018e9c0974aa14483d`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:04.931410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d4616a09-d722-4f12-a095-355d199fd927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:04.964430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54613e67-5e61-428a-96ed-64cb414e6c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:04.966796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54613e67-5e61-428a-96ed-64cb414e6c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:04.967449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54613e67-5e61-428a-96ed-64cb414e6c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:04.968075 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54613e67-5e61-428a-96ed-64cb414e6c47 admin admin] PATH_INFO: `/v3/projects/1170c2696d8c42638dc77928bcaa5164` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:04.970751 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54613e67-5e61-428a-96ed-64cb414e6c47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1170c2696d8c42638dc77928bcaa5164)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:04.975309 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54613e67-5e61-428a-96ed-64cb414e6c47 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:05.045081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1313/2629] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:05.063758 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4616a09-d722-4f12-a095-355d199fd927 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:05.067260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d4616a09-d722-4f12-a095-355d199fd927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:05.070887 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-54613e67-5e61-428a-96ed-64cb414e6c47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1170c2696d8c42638dc77928bcaa5164', 'request_id': 'req-54613e67-5e61-428a-96ed-64cb414e6c47'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:05.093629 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-54613e67-5e61-428a-96ed-64cb414e6c47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1170c2696d8c42638dc77928bcaa5164', 'request_id': 'req-54613e67-5e61-428a-96ed-64cb414e6c47'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:05.106470 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-54613e67-5e61-428a-96ed-64cb414e6c47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1170c2696d8c42638dc77928bcaa5164', 'request_id': 'req-54613e67-5e61-428a-96ed-64cb414e6c47'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:05.129327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-54613e67-5e61-428a-96ed-64cb414e6c47 admin admin] The token cache is being invalidate because project 1170c2696d8c42638dc77928bcaa5164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:51:05.130018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-54613e67-5e61-428a-96ed-64cb414e6c47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:05.130863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-54613e67-5e61-428a-96ed-64cb414e6c47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:05.132599 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1317/2630] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:51:04 2022] DELETE /identity/v3/projects/1170c2696d8c42638dc77928bcaa5164 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:51:05.142793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3869cc5-c336-4e82-a960-331773a1daa0 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:05.143434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3869cc5-c336-4e82-a960-331773a1daa0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:05.143794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3869cc5-c336-4e82-a960-331773a1daa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:05.158889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4616a09-d722-4f12-a095-355d199fd927 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:05.168775 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d3869cc5-c336-4e82-a960-331773a1daa0 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:05.170629 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d3869cc5-c336-4e82-a960-331773a1daa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:05.241319 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f991c-1aa4-4ab1-84bb-05a081fbff20 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '300fb5e1c5c546808b935871673e3d0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:05.243164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f991c-1aa4-4ab1-84bb-05a081fbff20 service glance] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:05.243702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f991c-1aa4-4ab1-84bb-05a081fbff20 service glance] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:05.244188 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f991c-1aa4-4ab1-84bb-05a081fbff20 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:05.244689 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:05.248222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-963f991c-1aa4-4ab1-84bb-05a081fbff20 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:05.273670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1318/2631] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:05.274868 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3869cc5-c336-4e82-a960-331773a1daa0 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:05.276788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d3869cc5-c336-4e82-a960-331773a1daa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:05.334569 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f991c-1aa4-4ab1-84bb-05a081fbff20 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:05.336882 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f991c-1aa4-4ab1-84bb-05a081fbff20 service glance] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:05.346583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3869cc5-c336-4e82-a960-331773a1daa0 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:05.347627 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1314/2632] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:05.348804 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da8f31-fa8a-4cab-a04a-fd5c1745e052 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:05.349100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da8f31-fa8a-4cab-a04a-fd5c1745e052 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:05.349378 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da8f31-fa8a-4cab-a04a-fd5c1745e052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:05.351333 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a866c-0f56-4b66-944a-24cc7d9ee0d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:05.352514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a866c-0f56-4b66-944a-24cc7d9ee0d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:05.352962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a866c-0f56-4b66-944a-24cc7d9ee0d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:05.353148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a866c-0f56-4b66-944a-24cc7d9ee0d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:05.355057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b46a866c-0f56-4b66-944a-24cc7d9ee0d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:05.368531 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:05.370113 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a866c-0f56-4b66-944a-24cc7d9ee0d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:05.371752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a866c-0f56-4b66-944a-24cc7d9ee0d4 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:05.377124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-39da8f31-fa8a-4cab-a04a-fd5c1745e052 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:05.378453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-39da8f31-fa8a-4cab-a04a-fd5c1745e052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:05.380785 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1319/2633] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:05.398131 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1315/2634] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:05.399540 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a866c-0f56-4b66-944a-24cc7d9ee0d4 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:05.419943 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a866c-0f56-4b66-944a-24cc7d9ee0d4 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:05.427160 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea6325-ddbc-4479-85b9-27dad885f8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:05.428127 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea6325-ddbc-4479-85b9-27dad885f8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:05.428570 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea6325-ddbc-4479-85b9-27dad885f8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:05.429049 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea6325-ddbc-4479-85b9-27dad885f8f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:05.430203 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea6325-ddbc-4479-85b9-27dad885f8f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:51:05.430664 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea6325-ddbc-4479-85b9-27dad885f8f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:05.432836 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea6325-ddbc-4479-85b9-27dad885f8f6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:05.457933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1320/2635] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:51:05.462936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da8f31-fa8a-4cab-a04a-fd5c1745e052 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:05.480442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da8f31-fa8a-4cab-a04a-fd5c1745e052 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:05.488203 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8049c-ea1a-4763-aa9f-067f389617f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:05.490025 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8049c-ea1a-4763-aa9f-067f389617f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:05.490662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8049c-ea1a-4763-aa9f-067f389617f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:05.491264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8049c-ea1a-4763-aa9f-067f389617f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:05.493345 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8049c-ea1a-4763-aa9f-067f389617f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:05.496974 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8049c-ea1a-4763-aa9f-067f389617f6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:05.563526 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1316/2636] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:51:05 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:51:05.569910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea6325-ddbc-4479-85b9-27dad885f8f6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:05.597906 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea6325-ddbc-4479-85b9-27dad885f8f6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:05.702298 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f4de1-4f61-4e5b-ba5f-3ba352e2fc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:05.703115 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f4de1-4f61-4e5b-ba5f-3ba352e2fc10 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:05.703280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f4de1-4f61-4e5b-ba5f-3ba352e2fc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:05.703463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f4de1-4f61-4e5b-ba5f-3ba352e2fc10 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:05.704147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f4de1-4f61-4e5b-ba5f-3ba352e2fc10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:05.706143 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f4de1-4f61-4e5b-ba5f-3ba352e2fc10 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:05.735769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1321/2637] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:51:05 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:51:05.740210 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8049c-ea1a-4763-aa9f-067f389617f6 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:05.751469 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8049c-ea1a-4763-aa9f-067f389617f6 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:05.756591 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960a27b-4882-41b4-926f-61f9ccb298bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:05.757249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960a27b-4882-41b4-926f-61f9ccb298bc admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:05.757619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960a27b-4882-41b4-926f-61f9ccb298bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:05.757924 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960a27b-4882-41b4-926f-61f9ccb298bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:05.758598 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960a27b-4882-41b4-926f-61f9ccb298bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:05.760264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960a27b-4882-41b4-926f-61f9ccb298bc admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:05.773179 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1317/2638] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:51:05.779179 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f4de1-4f61-4e5b-ba5f-3ba352e2fc10 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:05.802559 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f4de1-4f61-4e5b-ba5f-3ba352e2fc10 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:05.812216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138125d-f3a9-44e9-983a-ac35a0af4963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:05.813650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138125d-f3a9-44e9-983a-ac35a0af4963 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:05.814136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138125d-f3a9-44e9-983a-ac35a0af4963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:05.814649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138125d-f3a9-44e9-983a-ac35a0af4963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:05.817167 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e138125d-f3a9-44e9-983a-ac35a0af4963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:05.820960 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e138125d-f3a9-44e9-983a-ac35a0af4963 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:05.837188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1322/2639] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:51:05.844124 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960a27b-4882-41b4-926f-61f9ccb298bc admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:05.864512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960a27b-4882-41b4-926f-61f9ccb298bc admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:05.874237 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5cde7-c244-4af1-b5d6-3175c325f787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:05.875454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5cde7-c244-4af1-b5d6-3175c325f787 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:05.875764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5cde7-c244-4af1-b5d6-3175c325f787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:05.876234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5cde7-c244-4af1-b5d6-3175c325f787 admin admin] PATH_INFO: `/v3/projects/9dd20bb03bb540deb7a9854ad2f66ba0/users/1fd610351106467caec22791183a47d7/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:05.907360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5cde7-c244-4af1-b5d6-3175c325f787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dd20bb03bb540deb7a9854ad2f66ba0, user_id=1fd610351106467caec22791183a47d7, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:05.912384 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5cde7-c244-4af1-b5d6-3175c325f787 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:05.967106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1318/2640] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:51:05 2022] PUT /identity/v3/projects/9dd20bb03bb540deb7a9854ad2f66ba0/users/1fd610351106467caec22791183a47d7/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:51:06.045939 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2a5cf-3b2f-4eeb-92c1-9607f9f11525 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:06.046262 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2a5cf-3b2f-4eeb-92c1-9607f9f11525 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:06.046547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2a5cf-3b2f-4eeb-92c1-9607f9f11525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:06.085902 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c2b2a5cf-3b2f-4eeb-92c1-9607f9f11525 None None] MFA Rules not processed for user `1fd610351106467caec22791183a47d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:06.088983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c2b2a5cf-3b2f-4eeb-92c1-9607f9f11525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:06.183438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:06.214227 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1323/2641] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:06.231587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5cde7-c244-4af1-b5d6-3175c325f787 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:06.234651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0cf5cde7-c244-4af1-b5d6-3175c325f787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:06.329892 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5cde7-c244-4af1-b5d6-3175c325f787 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:06.334344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea333f-2581-4bae-932d-b784fc2c2e3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:06.335499 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea333f-2581-4bae-932d-b784fc2c2e3a service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:06.336007 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea333f-2581-4bae-932d-b784fc2c2e3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:06.337059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea333f-2581-4bae-932d-b784fc2c2e3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:06.349333 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea333f-2581-4bae-932d-b784fc2c2e3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:06.352829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea333f-2581-4bae-932d-b784fc2c2e3a service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:06.368592 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1319/2642] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:51:06.390258 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2a5cf-3b2f-4eeb-92c1-9607f9f11525 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:06.465820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2a5cf-3b2f-4eeb-92c1-9607f9f11525 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:06.473071 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e9ea8-a546-4791-9bcd-988816d397f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:06.474171 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e9ea8-a546-4791-9bcd-988816d397f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:06.474588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e9ea8-a546-4791-9bcd-988816d397f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:06.475109 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e9ea8-a546-4791-9bcd-988816d397f1 admin admin] PATH_INFO: `/v3/users/1fd610351106467caec22791183a47d7` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:06.477419 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e9ea8-a546-4791-9bcd-988816d397f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fd610351106467caec22791183a47d7)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:06.479519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e9ea8-a546-4791-9bcd-988816d397f1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:06.566782 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7f3e9ea8-a546-4791-9bcd-988816d397f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fd610351106467caec22791183a47d7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:06.584339 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7f3e9ea8-a546-4791-9bcd-988816d397f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fd610351106467caec22791183a47d7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:06.585004 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7f3e9ea8-a546-4791-9bcd-988816d397f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fd610351106467caec22791183a47d7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:06.616095 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7f3e9ea8-a546-4791-9bcd-988816d397f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fd610351106467caec22791183a47d7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:06.633896 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7f3e9ea8-a546-4791-9bcd-988816d397f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fd610351106467caec22791183a47d7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:06.641215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1324/2643] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:06 2022] DELETE /identity/v3/users/1fd610351106467caec22791183a47d7 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:06.647711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934936d2-4610-46e9-9069-bae7dacb5716 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:06.648185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934936d2-4610-46e9-9069-bae7dacb5716 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:06.648667 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934936d2-4610-46e9-9069-bae7dacb5716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:06.679350 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-934936d2-4610-46e9-9069-bae7dacb5716 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:06.681609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-934936d2-4610-46e9-9069-bae7dacb5716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:06.787946 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1320/2644] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:06.806150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e9ea8-a546-4791-9bcd-988816d397f1 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:06.808687 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7f3e9ea8-a546-4791-9bcd-988816d397f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:06.913191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e9ea8-a546-4791-9bcd-988816d397f1 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:06.916882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cf99e-64d3-4177-8e5d-5ec7309b4f27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:06.917549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cf99e-64d3-4177-8e5d-5ec7309b4f27 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:06.917835 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cf99e-64d3-4177-8e5d-5ec7309b4f27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:06.918112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cf99e-64d3-4177-8e5d-5ec7309b4f27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:06.929768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66cf99e-64d3-4177-8e5d-5ec7309b4f27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:06.931439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66cf99e-64d3-4177-8e5d-5ec7309b4f27 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:06.942837 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1325/2645] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:07.641790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934936d2-4610-46e9-9069-bae7dacb5716 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:07.644221 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-934936d2-4610-46e9-9069-bae7dacb5716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:07.694430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934936d2-4610-46e9-9069-bae7dacb5716 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:07.703953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bdbdec-d2ba-4b07-b2b7-2b629e933eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:07.705111 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdbdec-d2ba-4b07-b2b7-2b629e933eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:07.705695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdbdec-d2ba-4b07-b2b7-2b629e933eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:07.706179 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdbdec-d2ba-4b07-b2b7-2b629e933eeb admin admin] PATH_INFO: `/v3/projects/9dd20bb03bb540deb7a9854ad2f66ba0` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:07.708528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bdbdec-d2ba-4b07-b2b7-2b629e933eeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dd20bb03bb540deb7a9854ad2f66ba0)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:07.709165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb3c93-2046-4300-b9e7-ba6eef76f2f1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:07.709926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb3c93-2046-4300-b9e7-ba6eef76f2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:07.710327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb3c93-2046-4300-b9e7-ba6eef76f2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:07.712610 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bdbdec-d2ba-4b07-b2b7-2b629e933eeb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:07.746444 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c1cb3c93-2046-4300-b9e7-ba6eef76f2f1 None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:07.749076 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c1cb3c93-2046-4300-b9e7-ba6eef76f2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:07.760950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:07.847561 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:07.857400 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d9bdbdec-d2ba-4b07-b2b7-2b629e933eeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dd20bb03bb540deb7a9854ad2f66ba0', 'request_id': 'req-d9bdbdec-d2ba-4b07-b2b7-2b629e933eeb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:07.860615 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1326/2646] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 3 10:51:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:07.876222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d9bdbdec-d2ba-4b07-b2b7-2b629e933eeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dd20bb03bb540deb7a9854ad2f66ba0', 'request_id': 'req-d9bdbdec-d2ba-4b07-b2b7-2b629e933eeb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:07.891310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d9bdbdec-d2ba-4b07-b2b7-2b629e933eeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dd20bb03bb540deb7a9854ad2f66ba0', 'request_id': 'req-d9bdbdec-d2ba-4b07-b2b7-2b629e933eeb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:07.907253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d9bdbdec-d2ba-4b07-b2b7-2b629e933eeb admin admin] The token cache is being invalidate because project 9dd20bb03bb540deb7a9854ad2f66ba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:51:07.907550 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d9bdbdec-d2ba-4b07-b2b7-2b629e933eeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:07.908249 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d9bdbdec-d2ba-4b07-b2b7-2b629e933eeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:07.908746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb3c93-2046-4300-b9e7-ba6eef76f2f1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:07.910009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1321/2647] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:51:07 2022] DELETE /identity/v3/projects/9dd20bb03bb540deb7a9854ad2f66ba0 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:07.912573 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c1cb3c93-2046-4300-b9e7-ba6eef76f2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:07.915589 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94815a-91a1-47c2-ad91-8f5c04ad6614 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:07.915887 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94815a-91a1-47c2-ad91-8f5c04ad6614 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:07.916164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94815a-91a1-47c2-ad91-8f5c04ad6614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:07.945961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-1d94815a-91a1-47c2-ad91-8f5c04ad6614 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:07.953180 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1d94815a-91a1-47c2-ad91-8f5c04ad6614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:08.033076 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb3c93-2046-4300-b9e7-ba6eef76f2f1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:08.062151 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1322/2648] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:08.071049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bb874-3f48-4142-8e57-fa35ec8d460d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:08.071404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bb874-3f48-4142-8e57-fa35ec8d460d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:08.071687 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bb874-3f48-4142-8e57-fa35ec8d460d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:08.102390 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8f4bb874-3f48-4142-8e57-fa35ec8d460d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:08.103691 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8f4bb874-3f48-4142-8e57-fa35ec8d460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:08.112390 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1323/2649] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:08.117274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bb874-3f48-4142-8e57-fa35ec8d460d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:08.128493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bb874-3f48-4142-8e57-fa35ec8d460d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:08.135382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52191f00-7bd0-4864-acd7-4839aef9ea98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:08.135961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52191f00-7bd0-4864-acd7-4839aef9ea98 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:08.136336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52191f00-7bd0-4864-acd7-4839aef9ea98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:08.136625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52191f00-7bd0-4864-acd7-4839aef9ea98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:08.138470 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52191f00-7bd0-4864-acd7-4839aef9ea98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:51:08.138865 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52191f00-7bd0-4864-acd7-4839aef9ea98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:08.140411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52191f00-7bd0-4864-acd7-4839aef9ea98 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:08.141449 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d62b96-d958-4cff-84c3-75097245b086 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:08.142669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d62b96-d958-4cff-84c3-75097245b086 service swift] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:08.143248 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d62b96-d958-4cff-84c3-75097245b086 service swift] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:08.143847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d62b96-d958-4cff-84c3-75097245b086 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:08.147413 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-36d62b96-d958-4cff-84c3-75097245b086 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:08.157530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1324/2650] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:51:08.162514 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52191f00-7bd0-4864-acd7-4839aef9ea98 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:08.175388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52191f00-7bd0-4864-acd7-4839aef9ea98 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:08.180951 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ffbda-0f16-41a5-a17a-114d4bfae87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:08.181739 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ffbda-0f16-41a5-a17a-114d4bfae87b admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:08.182002 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ffbda-0f16-41a5-a17a-114d4bfae87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:08.182281 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ffbda-0f16-41a5-a17a-114d4bfae87b admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:08.183145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ffbda-0f16-41a5-a17a-114d4bfae87b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:08.185189 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ffbda-0f16-41a5-a17a-114d4bfae87b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:08.255171 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1325/2651] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:51:08 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:51:08.256908 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ffbda-0f16-41a5-a17a-114d4bfae87b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:08.306769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d62b96-d958-4cff-84c3-75097245b086 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:08.309443 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d62b96-d958-4cff-84c3-75097245b086 service swift] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:08.316953 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:08.334143 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1327/2652] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:51:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 10:51:08.435728 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ffbda-0f16-41a5-a17a-114d4bfae87b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:08.448582 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19240c6b-f50f-4e87-87c6-d40d3da078a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:08.451118 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19240c6b-f50f-4e87-87c6-d40d3da078a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:08.451833 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19240c6b-f50f-4e87-87c6-d40d3da078a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:08.452235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19240c6b-f50f-4e87-87c6-d40d3da078a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:08.454138 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19240c6b-f50f-4e87-87c6-d40d3da078a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:08.459634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19240c6b-f50f-4e87-87c6-d40d3da078a7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:08.506532 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1326/2653] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:51:08 2022] POST /identity/v3/users => generated 288 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:51:08.511541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d62b96-d958-4cff-84c3-75097245b086 service swift] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:08.548866 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d62b96-d958-4cff-84c3-75097245b086 service swift] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:08.575032 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0643fd-38f6-4cde-a013-c436c5e1ecd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:08.575910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0643fd-38f6-4cde-a013-c436c5e1ecd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:08.579615 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0643fd-38f6-4cde-a013-c436c5e1ecd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:08.580004 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0643fd-38f6-4cde-a013-c436c5e1ecd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:08.580811 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0643fd-38f6-4cde-a013-c436c5e1ecd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:08.587428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0643fd-38f6-4cde-a013-c436c5e1ecd3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:08.644891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1328/2654] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:51:08.649856 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19240c6b-f50f-4e87-87c6-d40d3da078a7 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:08.700101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19240c6b-f50f-4e87-87c6-d40d3da078a7 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:08.726725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06a8b8-0b5b-4801-91d9-52c5eb085c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:08.730266 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06a8b8-0b5b-4801-91d9-52c5eb085c26 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:08.730725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06a8b8-0b5b-4801-91d9-52c5eb085c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:08.731046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06a8b8-0b5b-4801-91d9-52c5eb085c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:08.731778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06a8b8-0b5b-4801-91d9-52c5eb085c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:08.738842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06a8b8-0b5b-4801-91d9-52c5eb085c26 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:08.763440 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1327/2655] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:51:08.769482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0643fd-38f6-4cde-a013-c436c5e1ecd3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:08.793607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0643fd-38f6-4cde-a013-c436c5e1ecd3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:08.806534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056a00a-b6ba-451e-9fe1-ab3bab2d4ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:08.807342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056a00a-b6ba-451e-9fe1-ab3bab2d4ad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:08.809148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056a00a-b6ba-451e-9fe1-ab3bab2d4ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:08.809614 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056a00a-b6ba-451e-9fe1-ab3bab2d4ad5 admin admin] PATH_INFO: `/v3/projects/3657fbe527914efbbcaa764344896060/users/41ebd0e7484b4a7d801979851ab79638/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:08.849045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056a00a-b6ba-451e-9fe1-ab3bab2d4ad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3657fbe527914efbbcaa764344896060, user_id=41ebd0e7484b4a7d801979851ab79638, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:08.878537 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056a00a-b6ba-451e-9fe1-ab3bab2d4ad5 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:08.925880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1329/2656] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:51:08 2022] PUT /identity/v3/projects/3657fbe527914efbbcaa764344896060/users/41ebd0e7484b4a7d801979851ab79638/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:51:08.948368 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4aad9a-5b07-4146-8541-92cd4fd3606f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:08.949563 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4aad9a-5b07-4146-8541-92cd4fd3606f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:08.950091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4aad9a-5b07-4146-8541-92cd4fd3606f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:08.990263 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:09.027001 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8e4aad9a-5b07-4146-8541-92cd4fd3606f None None] MFA Rules not processed for user `41ebd0e7484b4a7d801979851ab79638`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:09.028856 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8e4aad9a-5b07-4146-8541-92cd4fd3606f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:09.231006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1328/2657] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:09.258503 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056a00a-b6ba-451e-9fe1-ab3bab2d4ad5 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:09.264827 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7056a00a-b6ba-451e-9fe1-ab3bab2d4ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:09.524723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056a00a-b6ba-451e-9fe1-ab3bab2d4ad5 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:09.538515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea0286-f54d-405b-b255-524d4f30fc94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:09.539070 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea0286-f54d-405b-b255-524d4f30fc94 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:09.539924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea0286-f54d-405b-b255-524d4f30fc94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:09.540245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea0286-f54d-405b-b255-524d4f30fc94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:09.550802 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea0286-f54d-405b-b255-524d4f30fc94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:09.552291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea0286-f54d-405b-b255-524d4f30fc94 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:09.575232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1330/2658] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:13.822424 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4aad9a-5b07-4146-8541-92cd4fd3606f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:13.921130 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4aad9a-5b07-4146-8541-92cd4fd3606f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:13.926641 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587e10a-63d5-4d59-9b5c-31f6f691de05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:13.927403 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587e10a-63d5-4d59-9b5c-31f6f691de05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:13.927693 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587e10a-63d5-4d59-9b5c-31f6f691de05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:13.928062 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587e10a-63d5-4d59-9b5c-31f6f691de05 admin admin] PATH_INFO: `/v3/users/41ebd0e7484b4a7d801979851ab79638` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:13.929435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587e10a-63d5-4d59-9b5c-31f6f691de05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ebd0e7484b4a7d801979851ab79638)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:13.932349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587e10a-63d5-4d59-9b5c-31f6f691de05 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:14.003511 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea0286-f54d-405b-b255-524d4f30fc94 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:14.005925 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d3ea0286-f54d-405b-b255-524d4f30fc94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:14.016216 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2587e10a-63d5-4d59-9b5c-31f6f691de05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ebd0e7484b4a7d801979851ab79638'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:14.026078 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2587e10a-63d5-4d59-9b5c-31f6f691de05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ebd0e7484b4a7d801979851ab79638'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:14.035496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2587e10a-63d5-4d59-9b5c-31f6f691de05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ebd0e7484b4a7d801979851ab79638'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:14.036458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2587e10a-63d5-4d59-9b5c-31f6f691de05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ebd0e7484b4a7d801979851ab79638'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:14.037399 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2587e10a-63d5-4d59-9b5c-31f6f691de05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ebd0e7484b4a7d801979851ab79638'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:14.072075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1329/2659] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:13 2022] DELETE /identity/v3/users/41ebd0e7484b4a7d801979851ab79638 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:14.079418 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca34c9e-2c8b-4d19-bbd7-395bb7ecd433 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:14.079712 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca34c9e-2c8b-4d19-bbd7-395bb7ecd433 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:14.080006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca34c9e-2c8b-4d19-bbd7-395bb7ecd433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:14.091902 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea0286-f54d-405b-b255-524d4f30fc94 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:14.103730 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-bca34c9e-2c8b-4d19-bbd7-395bb7ecd433 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:14.118708 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bca34c9e-2c8b-4d19-bbd7-395bb7ecd433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:14.133586 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:14.158029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0add6d4-e9c0-46bf-8623-563c192a56ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:14.158683 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0add6d4-e9c0-46bf-8623-563c192a56ae service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:14.158996 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0add6d4-e9c0-46bf-8623-563c192a56ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:14.159274 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0add6d4-e9c0-46bf-8623-563c192a56ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:14.161216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d0add6d4-e9c0-46bf-8623-563c192a56ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:14.212836 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1330/2660] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:14.220664 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:14.223699 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca34c9e-2c8b-4d19-bbd7-395bb7ecd433 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:14.225205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bca34c9e-2c8b-4d19-bbd7-395bb7ecd433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:14.240967 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca34c9e-2c8b-4d19-bbd7-395bb7ecd433 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:14.246047 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469df5d-50aa-446b-8898-5cc42965ef3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:14.246878 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469df5d-50aa-446b-8898-5cc42965ef3c service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:14.247163 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469df5d-50aa-446b-8898-5cc42965ef3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:14.247444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469df5d-50aa-446b-8898-5cc42965ef3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:14.257960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469df5d-50aa-446b-8898-5cc42965ef3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:14.258927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0add6d4-e9c0-46bf-8623-563c192a56ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:14.259432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469df5d-50aa-446b-8898-5cc42965ef3c service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:14.260541 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0add6d4-e9c0-46bf-8623-563c192a56ae service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:14.270256 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1331/2661] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:51:14.271368 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1331/2662] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:51:15.020884 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469df5d-50aa-446b-8898-5cc42965ef3c service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:15.022815 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a469df5d-50aa-446b-8898-5cc42965ef3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:15.070757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469df5d-50aa-446b-8898-5cc42965ef3c service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:15.091316 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629eba0-5f5c-442b-8e95-7ffdabbff285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:15.092245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629eba0-5f5c-442b-8e95-7ffdabbff285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:15.092568 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629eba0-5f5c-442b-8e95-7ffdabbff285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:15.092955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629eba0-5f5c-442b-8e95-7ffdabbff285 admin admin] PATH_INFO: `/v3/projects/3657fbe527914efbbcaa764344896060` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:15.094824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629eba0-5f5c-442b-8e95-7ffdabbff285 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3657fbe527914efbbcaa764344896060)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:15.097724 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629eba0-5f5c-442b-8e95-7ffdabbff285 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:15.216362 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a629eba0-5f5c-442b-8e95-7ffdabbff285 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3657fbe527914efbbcaa764344896060', 'request_id': 'req-a629eba0-5f5c-442b-8e95-7ffdabbff285'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:15.225827 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a629eba0-5f5c-442b-8e95-7ffdabbff285 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3657fbe527914efbbcaa764344896060', 'request_id': 'req-a629eba0-5f5c-442b-8e95-7ffdabbff285'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:15.238396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a629eba0-5f5c-442b-8e95-7ffdabbff285 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3657fbe527914efbbcaa764344896060', 'request_id': 'req-a629eba0-5f5c-442b-8e95-7ffdabbff285'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:15.252994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a629eba0-5f5c-442b-8e95-7ffdabbff285 admin admin] The token cache is being invalidate because project 3657fbe527914efbbcaa764344896060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:51:15.253780 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a629eba0-5f5c-442b-8e95-7ffdabbff285 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:15.254560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a629eba0-5f5c-442b-8e95-7ffdabbff285 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:15.256748 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1332/2663] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:51:15 2022] DELETE /identity/v3/projects/3657fbe527914efbbcaa764344896060 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:15.262232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9401d5-21da-467c-a217-64838cf0e1d9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:15.262919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9401d5-21da-467c-a217-64838cf0e1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:15.262919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9401d5-21da-467c-a217-64838cf0e1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:15.291983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2a9401d5-21da-467c-a217-64838cf0e1d9 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:15.293820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2a9401d5-21da-467c-a217-64838cf0e1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:15.408605 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1332/2664] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:15.418726 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d79d3-f5bf-48ee-8e3b-70899fc5da81 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:15.418726 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d79d3-f5bf-48ee-8e3b-70899fc5da81 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:15.419854 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d79d3-f5bf-48ee-8e3b-70899fc5da81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:15.456055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f56d79d3-f5bf-48ee-8e3b-70899fc5da81 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:15.466015 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f56d79d3-f5bf-48ee-8e3b-70899fc5da81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:15.475449 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1333/2665] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:15.479755 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9401d5-21da-467c-a217-64838cf0e1d9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:15.490447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9401d5-21da-467c-a217-64838cf0e1d9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:15.496524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84af3ef-223e-4b3d-bb22-985ca459382c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:15.497612 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84af3ef-223e-4b3d-bb22-985ca459382c admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:15.498102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84af3ef-223e-4b3d-bb22-985ca459382c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:15.498737 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84af3ef-223e-4b3d-bb22-985ca459382c admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:15.499641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84af3ef-223e-4b3d-bb22-985ca459382c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:51:15.500151 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84af3ef-223e-4b3d-bb22-985ca459382c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:15.502010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84af3ef-223e-4b3d-bb22-985ca459382c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:15.519928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1333/2666] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:51:15.532059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d79d3-f5bf-48ee-8e3b-70899fc5da81 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:15.565436 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d79d3-f5bf-48ee-8e3b-70899fc5da81 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:15.577911 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa73e4-b845-437c-aec5-7cb565812793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:15.578751 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa73e4-b845-437c-aec5-7cb565812793 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:15.579042 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa73e4-b845-437c-aec5-7cb565812793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:15.579319 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa73e4-b845-437c-aec5-7cb565812793 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:15.580012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa73e4-b845-437c-aec5-7cb565812793 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:15.582720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa73e4-b845-437c-aec5-7cb565812793 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:15.620707 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1334/2667] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:51:15 2022] POST /identity/v3/projects => generated 367 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:51:15.624699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84af3ef-223e-4b3d-bb22-985ca459382c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:15.636185 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84af3ef-223e-4b3d-bb22-985ca459382c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:15.823090 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edecdd4-adb2-4ff5-8438-ba24e03ab0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:15.823737 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edecdd4-adb2-4ff5-8438-ba24e03ab0eb admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:15.824027 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edecdd4-adb2-4ff5-8438-ba24e03ab0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:15.824423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edecdd4-adb2-4ff5-8438-ba24e03ab0eb admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:15.825110 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edecdd4-adb2-4ff5-8438-ba24e03ab0eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:15.827058 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edecdd4-adb2-4ff5-8438-ba24e03ab0eb admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:15.905948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1334/2668] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:51:15 2022] POST /identity/v3/users => generated 284 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:51:15.906469 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa73e4-b845-437c-aec5-7cb565812793 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:15.926820 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa73e4-b845-437c-aec5-7cb565812793 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:15.931809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f595a-ebc4-4efd-93aa-17553bf050fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:15.932451 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f595a-ebc4-4efd-93aa-17553bf050fe admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:15.932757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f595a-ebc4-4efd-93aa-17553bf050fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:15.933012 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f595a-ebc4-4efd-93aa-17553bf050fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:15.933765 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f595a-ebc4-4efd-93aa-17553bf050fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:15.935399 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f595a-ebc4-4efd-93aa-17553bf050fe admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:15.968412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edecdd4-adb2-4ff5-8438-ba24e03ab0eb admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:15.974826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1335/2669] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:51:15.984008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edecdd4-adb2-4ff5-8438-ba24e03ab0eb admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:15.995407 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bb8fc-6cf9-4cff-994c-d69b5d98759f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:15.996042 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bb8fc-6cf9-4cff-994c-d69b5d98759f admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:15.996321 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bb8fc-6cf9-4cff-994c-d69b5d98759f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:15.996813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bb8fc-6cf9-4cff-994c-d69b5d98759f admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:15.997571 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bb8fc-6cf9-4cff-994c-d69b5d98759f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:16.000519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bb8fc-6cf9-4cff-994c-d69b5d98759f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:16.019086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1335/2670] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:51:16.020332 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f595a-ebc4-4efd-93aa-17553bf050fe admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:16.043030 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f595a-ebc4-4efd-93aa-17553bf050fe admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:16.053410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23daa25f-0934-4a0e-8a4d-07ae91ae3325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:16.054132 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23daa25f-0934-4a0e-8a4d-07ae91ae3325 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:16.054626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23daa25f-0934-4a0e-8a4d-07ae91ae3325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:16.055570 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23daa25f-0934-4a0e-8a4d-07ae91ae3325 admin admin] PATH_INFO: `/v3/projects/46f6e58889084b2ab7cb010c9c9c704f/users/7aa23dc4d6614cedbd22db551682860e/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:16.085873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23daa25f-0934-4a0e-8a4d-07ae91ae3325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f6e58889084b2ab7cb010c9c9c704f, user_id=7aa23dc4d6614cedbd22db551682860e, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:16.090556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23daa25f-0934-4a0e-8a4d-07ae91ae3325 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:16.117877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1336/2671] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:51:16 2022] PUT /identity/v3/projects/46f6e58889084b2ab7cb010c9c9c704f/users/7aa23dc4d6614cedbd22db551682860e/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:16.128572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24951a8b-c420-4102-a040-eb9737d423b7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:16.128865 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24951a8b-c420-4102-a040-eb9737d423b7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:16.128975 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24951a8b-c420-4102-a040-eb9737d423b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:16.185580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-24951a8b-c420-4102-a040-eb9737d423b7 None None] MFA Rules not processed for user `7aa23dc4d6614cedbd22db551682860e`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:16.185580 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-24951a8b-c420-4102-a040-eb9737d423b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:16.245779 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:16.294641 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1336/2672] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:16.452645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23daa25f-0934-4a0e-8a4d-07ae91ae3325 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:16.454601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-23daa25f-0934-4a0e-8a4d-07ae91ae3325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:16.525234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:16.581144 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23daa25f-0934-4a0e-8a4d-07ae91ae3325 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:16.586766 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd5e92-7103-4f36-829f-533fb778b263 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '300fb5e1c5c546808b935871673e3d0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:16.587708 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd5e92-7103-4f36-829f-533fb778b263 service glance] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:16.588173 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd5e92-7103-4f36-829f-533fb778b263 service glance] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:16.588612 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd5e92-7103-4f36-829f-533fb778b263 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:16.616264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd5e92-7103-4f36-829f-533fb778b263 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:16.618619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd5e92-7103-4f36-829f-533fb778b263 service glance] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:16.638216 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1337/2673] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:23.779890 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24951a8b-c420-4102-a040-eb9737d423b7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:23.782136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-24951a8b-c420-4102-a040-eb9737d423b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:23.869131 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24951a8b-c420-4102-a040-eb9737d423b7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:23.875341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777123d-ed58-43ef-9d48-f66732f85753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:23.876353 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777123d-ed58-43ef-9d48-f66732f85753 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:23.876756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777123d-ed58-43ef-9d48-f66732f85753 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:23.877132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777123d-ed58-43ef-9d48-f66732f85753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:23.879281 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8777123d-ed58-43ef-9d48-f66732f85753 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:23.974012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777123d-ed58-43ef-9d48-f66732f85753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:23.975619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777123d-ed58-43ef-9d48-f66732f85753 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:23.987700 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1337/2674] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:23.996864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd5e92-7103-4f36-829f-533fb778b263 service glance] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:23.998518 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-aabd5e92-7103-4f36-829f-533fb778b263 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:24.013787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd5e92-7103-4f36-829f-533fb778b263 service glance] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:24.019473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97033bb5-30c2-40ea-933e-f6c6842fc083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:24.020600 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97033bb5-30c2-40ea-933e-f6c6842fc083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:24.021084 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97033bb5-30c2-40ea-933e-f6c6842fc083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:24.024954 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97033bb5-30c2-40ea-933e-f6c6842fc083 admin admin] PATH_INFO: `/v3/users/13e30ef7b13a48018e9c0974aa14483d` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:24.024954 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97033bb5-30c2-40ea-933e-f6c6842fc083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13e30ef7b13a48018e9c0974aa14483d)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:24.026819 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97033bb5-30c2-40ea-933e-f6c6842fc083 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:24.099260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-97033bb5-30c2-40ea-933e-f6c6842fc083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13e30ef7b13a48018e9c0974aa14483d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:24.110147 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-97033bb5-30c2-40ea-933e-f6c6842fc083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13e30ef7b13a48018e9c0974aa14483d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:24.121036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-97033bb5-30c2-40ea-933e-f6c6842fc083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13e30ef7b13a48018e9c0974aa14483d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:24.122346 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-97033bb5-30c2-40ea-933e-f6c6842fc083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13e30ef7b13a48018e9c0974aa14483d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:24.123424 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-97033bb5-30c2-40ea-933e-f6c6842fc083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13e30ef7b13a48018e9c0974aa14483d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:24.152590 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1338/2675] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:23 2022] DELETE /identity/v3/users/13e30ef7b13a48018e9c0974aa14483d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:51:24.157520 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168adc3-67b3-41fa-b929-ed6488959d0c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:24.157989 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168adc3-67b3-41fa-b929-ed6488959d0c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:24.158410 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168adc3-67b3-41fa-b929-ed6488959d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:24.180804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9168adc3-67b3-41fa-b929-ed6488959d0c None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:24.182170 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9168adc3-67b3-41fa-b929-ed6488959d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:24.279928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1338/2676] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:24.294344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97033bb5-30c2-40ea-933e-f6c6842fc083 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:24.296239 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-97033bb5-30c2-40ea-933e-f6c6842fc083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:24.384809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97033bb5-30c2-40ea-933e-f6c6842fc083 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:24.388961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007965bd-5316-47b8-a99e-aea4770e3074 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:24.389732 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007965bd-5316-47b8-a99e-aea4770e3074 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:24.390129 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007965bd-5316-47b8-a99e-aea4770e3074 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:24.390468 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007965bd-5316-47b8-a99e-aea4770e3074 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:24.403154 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007965bd-5316-47b8-a99e-aea4770e3074 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:24.403408 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007965bd-5316-47b8-a99e-aea4770e3074 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:24.417157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1339/2677] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:51:25.570756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168adc3-67b3-41fa-b929-ed6488959d0c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:25.572285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9168adc3-67b3-41fa-b929-ed6488959d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:25.587146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168adc3-67b3-41fa-b929-ed6488959d0c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:25.594316 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e82a4-49b4-4287-8502-c026813cbd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:25.595483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e82a4-49b4-4287-8502-c026813cbd67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:25.596151 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e82a4-49b4-4287-8502-c026813cbd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:25.596616 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e82a4-49b4-4287-8502-c026813cbd67 admin admin] PATH_INFO: `/v3/projects/805b8544ac5c4f9b96f6bb0435e30574` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:25.599348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e82a4-49b4-4287-8502-c026813cbd67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=805b8544ac5c4f9b96f6bb0435e30574)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:25.605662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e82a4-49b4-4287-8502-c026813cbd67 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:25.635849 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:25.706601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ea4e82a4-49b4-4287-8502-c026813cbd67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '805b8544ac5c4f9b96f6bb0435e30574', 'request_id': 'req-ea4e82a4-49b4-4287-8502-c026813cbd67'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:25.734237 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ea4e82a4-49b4-4287-8502-c026813cbd67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '805b8544ac5c4f9b96f6bb0435e30574', 'request_id': 'req-ea4e82a4-49b4-4287-8502-c026813cbd67'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:25.749218 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ea4e82a4-49b4-4287-8502-c026813cbd67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '805b8544ac5c4f9b96f6bb0435e30574', 'request_id': 'req-ea4e82a4-49b4-4287-8502-c026813cbd67'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:25.763923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ea4e82a4-49b4-4287-8502-c026813cbd67 admin admin] The token cache is being invalidate because project 805b8544ac5c4f9b96f6bb0435e30574 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:51:25.765598 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ea4e82a4-49b4-4287-8502-c026813cbd67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:25.768369 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ea4e82a4-49b4-4287-8502-c026813cbd67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:25.771238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1339/2678] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:51:25 2022] DELETE /identity/v3/projects/805b8544ac5c4f9b96f6bb0435e30574 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:25.778881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba214dc-e051-498e-b0d7-323aea82bf2c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:25.779552 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba214dc-e051-498e-b0d7-323aea82bf2c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:25.779552 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba214dc-e051-498e-b0d7-323aea82bf2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:25.802158 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-bba214dc-e051-498e-b0d7-323aea82bf2c None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:25.803464 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bba214dc-e051-498e-b0d7-323aea82bf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:25.878397 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:25.965515 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1340/2679] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:25.974680 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71c73a-3b77-4c1e-ab76-04f352223b05 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:25.975061 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71c73a-3b77-4c1e-ab76-04f352223b05 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:25.975240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71c73a-3b77-4c1e-ab76-04f352223b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:26.045458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2f71c73a-3b77-4c1e-ab76-04f352223b05 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:26.051271 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2f71c73a-3b77-4c1e-ab76-04f352223b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:26.080807 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1340/2680] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:26.089261 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba214dc-e051-498e-b0d7-323aea82bf2c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:26.105311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba214dc-e051-498e-b0d7-323aea82bf2c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:26.138624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cd18e-eab9-451e-a950-777c17082025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:26.140291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cd18e-eab9-451e-a950-777c17082025 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:26.141099 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cd18e-eab9-451e-a950-777c17082025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:26.152893 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cd18e-eab9-451e-a950-777c17082025 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:26.156022 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cd18e-eab9-451e-a950-777c17082025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:51:26.157040 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cd18e-eab9-451e-a950-777c17082025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:26.161395 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cd18e-eab9-451e-a950-777c17082025 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:26.193459 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1341/2681] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:51:26.208218 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71c73a-3b77-4c1e-ab76-04f352223b05 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:26.240771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71c73a-3b77-4c1e-ab76-04f352223b05 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:26.250093 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981b72f-fae4-4918-866b-46d96cfccf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:26.251218 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981b72f-fae4-4918-866b-46d96cfccf91 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:26.252174 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981b72f-fae4-4918-866b-46d96cfccf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:26.254119 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981b72f-fae4-4918-866b-46d96cfccf91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:26.256798 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981b72f-fae4-4918-866b-46d96cfccf91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:26.266820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981b72f-fae4-4918-866b-46d96cfccf91 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:26.355436 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1341/2682] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:51:26 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:51:26.358987 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cd18e-eab9-451e-a950-777c17082025 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:26.377200 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cd18e-eab9-451e-a950-777c17082025 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:26.484788 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43293a0-4a54-42dd-b5bd-e45d7347061b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:26.489394 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43293a0-4a54-42dd-b5bd-e45d7347061b admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:26.489877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43293a0-4a54-42dd-b5bd-e45d7347061b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:26.491781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43293a0-4a54-42dd-b5bd-e45d7347061b admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:26.497323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43293a0-4a54-42dd-b5bd-e45d7347061b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:26.499720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43293a0-4a54-42dd-b5bd-e45d7347061b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:26.533262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1342/2683] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:51:26 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:51:26.540422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981b72f-fae4-4918-866b-46d96cfccf91 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:26.559181 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981b72f-fae4-4918-866b-46d96cfccf91 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:26.580763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5fe7c-fc28-4a9b-a352-0d6d030cc962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:26.581623 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5fe7c-fc28-4a9b-a352-0d6d030cc962 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:26.582020 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5fe7c-fc28-4a9b-a352-0d6d030cc962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:26.582495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5fe7c-fc28-4a9b-a352-0d6d030cc962 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:26.583632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5fe7c-fc28-4a9b-a352-0d6d030cc962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:26.588919 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5fe7c-fc28-4a9b-a352-0d6d030cc962 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:26.645422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1342/2684] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:51:26.651995 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43293a0-4a54-42dd-b5bd-e45d7347061b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:26.681595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43293a0-4a54-42dd-b5bd-e45d7347061b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:26.703782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396c851-12f5-46f4-aa17-6bca978dd5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:26.704583 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396c851-12f5-46f4-aa17-6bca978dd5ce admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:26.705276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396c851-12f5-46f4-aa17-6bca978dd5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:26.705276 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396c851-12f5-46f4-aa17-6bca978dd5ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:26.706094 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396c851-12f5-46f4-aa17-6bca978dd5ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:26.714902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396c851-12f5-46f4-aa17-6bca978dd5ce admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:26.732379 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1343/2685] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:51:26.739136 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5fe7c-fc28-4a9b-a352-0d6d030cc962 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:26.763135 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5fe7c-fc28-4a9b-a352-0d6d030cc962 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:26.777380 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25817880-66b8-4fe6-b8fd-83a3c784d82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:26.778650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25817880-66b8-4fe6-b8fd-83a3c784d82c admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:26.778650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25817880-66b8-4fe6-b8fd-83a3c784d82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:26.779706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25817880-66b8-4fe6-b8fd-83a3c784d82c admin admin] PATH_INFO: `/v3/projects/ee7b3e4b8952437f8280526c5eb40080/users/9b58410a7c9e4a5ea060588a8c948b23/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:26.798964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25817880-66b8-4fe6-b8fd-83a3c784d82c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7b3e4b8952437f8280526c5eb40080, user_id=9b58410a7c9e4a5ea060588a8c948b23, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:26.802022 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25817880-66b8-4fe6-b8fd-83a3c784d82c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:26.849687 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1343/2686] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:51:26 2022] PUT /identity/v3/projects/ee7b3e4b8952437f8280526c5eb40080/users/9b58410a7c9e4a5ea060588a8c948b23/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:51:27.012000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acc27e-2fbb-4764-a35f-a6679742cc07 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:27.012000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acc27e-2fbb-4764-a35f-a6679742cc07 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:27.012000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acc27e-2fbb-4764-a35f-a6679742cc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:27.180143 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-31acc27e-2fbb-4764-a35f-a6679742cc07 None None] MFA Rules not processed for user `9b58410a7c9e4a5ea060588a8c948b23`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:27.184838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-31acc27e-2fbb-4764-a35f-a6679742cc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:27.403834 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1344/2687] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:27.419693 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25817880-66b8-4fe6-b8fd-83a3c784d82c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:27.422403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-25817880-66b8-4fe6-b8fd-83a3c784d82c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:27.560897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25817880-66b8-4fe6-b8fd-83a3c784d82c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:27.573569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14607f9-feb3-43bc-9d2e-c228efb1b136 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:27.574425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14607f9-feb3-43bc-9d2e-c228efb1b136 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:27.575386 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14607f9-feb3-43bc-9d2e-c228efb1b136 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:27.575681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14607f9-feb3-43bc-9d2e-c228efb1b136 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:27.602310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14607f9-feb3-43bc-9d2e-c228efb1b136 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:27.603898 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14607f9-feb3-43bc-9d2e-c228efb1b136 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:27.615156 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1344/2688] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:30.598213 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31acc27e-2fbb-4764-a35f-a6679742cc07 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:30.702982 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:30.745897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31acc27e-2fbb-4764-a35f-a6679742cc07 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:30.756077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba40ed0-5a88-4d04-98c5-5353a0f2f0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:30.760273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba40ed0-5a88-4d04-98c5-5353a0f2f0df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:30.761690 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba40ed0-5a88-4d04-98c5-5353a0f2f0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:30.762772 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba40ed0-5a88-4d04-98c5-5353a0f2f0df admin admin] PATH_INFO: `/v3/users/9b58410a7c9e4a5ea060588a8c948b23` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:30.765601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba40ed0-5a88-4d04-98c5-5353a0f2f0df admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b58410a7c9e4a5ea060588a8c948b23)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:30.771689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba40ed0-5a88-4d04-98c5-5353a0f2f0df admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:30.893994 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1ba40ed0-5a88-4d04-98c5-5353a0f2f0df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b58410a7c9e4a5ea060588a8c948b23'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:30.903479 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1ba40ed0-5a88-4d04-98c5-5353a0f2f0df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b58410a7c9e4a5ea060588a8c948b23'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:30.922291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1ba40ed0-5a88-4d04-98c5-5353a0f2f0df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b58410a7c9e4a5ea060588a8c948b23'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:30.922291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1ba40ed0-5a88-4d04-98c5-5353a0f2f0df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b58410a7c9e4a5ea060588a8c948b23'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:30.922291 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1ba40ed0-5a88-4d04-98c5-5353a0f2f0df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b58410a7c9e4a5ea060588a8c948b23'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:30.937787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1345/2689] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:30 2022] DELETE /identity/v3/users/9b58410a7c9e4a5ea060588a8c948b23 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:30.943024 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6273107-a1e9-4fda-8bd8-f660e04f4da8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:30.943868 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6273107-a1e9-4fda-8bd8-f660e04f4da8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:30.944018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6273107-a1e9-4fda-8bd8-f660e04f4da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:30.971769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a6273107-a1e9-4fda-8bd8-f660e04f4da8 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:30.973219 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a6273107-a1e9-4fda-8bd8-f660e04f4da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:31.030450 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:31.072107 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1345/2690] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:31.084160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba40ed0-5a88-4d04-98c5-5353a0f2f0df admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:31.085844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1ba40ed0-5a88-4d04-98c5-5353a0f2f0df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:31.183235 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba40ed0-5a88-4d04-98c5-5353a0f2f0df admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:31.187909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a70604-125b-4bad-bb93-55b194a20e2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:31.188697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a70604-125b-4bad-bb93-55b194a20e2a service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:31.189049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a70604-125b-4bad-bb93-55b194a20e2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:31.189404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a70604-125b-4bad-bb93-55b194a20e2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:31.205232 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a70604-125b-4bad-bb93-55b194a20e2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:31.206025 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a70604-125b-4bad-bb93-55b194a20e2a service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:31.217787 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1346/2691] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:31.755242 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6273107-a1e9-4fda-8bd8-f660e04f4da8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:31.757775 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a6273107-a1e9-4fda-8bd8-f660e04f4da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:31.792256 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6273107-a1e9-4fda-8bd8-f660e04f4da8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:31.806672 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458e858-bee4-441c-8e11-b62ef08e2525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:31.811487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458e858-bee4-441c-8e11-b62ef08e2525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:31.812964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458e858-bee4-441c-8e11-b62ef08e2525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:31.813947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458e858-bee4-441c-8e11-b62ef08e2525 admin admin] PATH_INFO: `/v3/projects/ee7b3e4b8952437f8280526c5eb40080` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:31.818126 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458e858-bee4-441c-8e11-b62ef08e2525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee7b3e4b8952437f8280526c5eb40080)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:31.823086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458e858-bee4-441c-8e11-b62ef08e2525 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:31.924913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6458e858-bee4-441c-8e11-b62ef08e2525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee7b3e4b8952437f8280526c5eb40080', 'request_id': 'req-6458e858-bee4-441c-8e11-b62ef08e2525'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:31.943724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6458e858-bee4-441c-8e11-b62ef08e2525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee7b3e4b8952437f8280526c5eb40080', 'request_id': 'req-6458e858-bee4-441c-8e11-b62ef08e2525'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:31.955347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6458e858-bee4-441c-8e11-b62ef08e2525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee7b3e4b8952437f8280526c5eb40080', 'request_id': 'req-6458e858-bee4-441c-8e11-b62ef08e2525'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:31.969607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6458e858-bee4-441c-8e11-b62ef08e2525 admin admin] The token cache is being invalidate because project ee7b3e4b8952437f8280526c5eb40080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:51:31.969853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6458e858-bee4-441c-8e11-b62ef08e2525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:31.970602 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6458e858-bee4-441c-8e11-b62ef08e2525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:31.972654 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1346/2692] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:51:31 2022] DELETE /identity/v3/projects/ee7b3e4b8952437f8280526c5eb40080 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:51:31.979558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e2174-5fbb-461b-9d28-e28926666531 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:31.980197 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e2174-5fbb-461b-9d28-e28926666531 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:31.980543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e2174-5fbb-461b-9d28-e28926666531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:32.043810 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-dc2e2174-5fbb-461b-9d28-e28926666531 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:32.046566 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-dc2e2174-5fbb-461b-9d28-e28926666531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:32.153207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:32.179112 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315694d3-2742-4cc0-b04c-99cfebeec93f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:32.179418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315694d3-2742-4cc0-b04c-99cfebeec93f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:32.180260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315694d3-2742-4cc0-b04c-99cfebeec93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:32.187529 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1347/2693] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:32.308836 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-315694d3-2742-4cc0-b04c-99cfebeec93f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:32.312221 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-315694d3-2742-4cc0-b04c-99cfebeec93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:32.424205 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1347/2694] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:32.429305 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e2174-5fbb-461b-9d28-e28926666531 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:32.573877 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e2174-5fbb-461b-9d28-e28926666531 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:32.580661 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e1c62-2ed8-4e55-8a33-19479e4a3c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:32.581544 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e1c62-2ed8-4e55-8a33-19479e4a3c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:32.581932 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e1c62-2ed8-4e55-8a33-19479e4a3c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:32.588547 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e1c62-2ed8-4e55-8a33-19479e4a3c97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:32.588547 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e1c62-2ed8-4e55-8a33-19479e4a3c97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:51:32.588547 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e1c62-2ed8-4e55-8a33-19479e4a3c97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:32.588547 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e1c62-2ed8-4e55-8a33-19479e4a3c97 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:32.607212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315694d3-2742-4cc0-b04c-99cfebeec93f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:32.607212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-315694d3-2742-4cc0-b04c-99cfebeec93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:32.639046 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1348/2695] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:51:32.641972 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e1c62-2ed8-4e55-8a33-19479e4a3c97 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:32.677933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e1c62-2ed8-4e55-8a33-19479e4a3c97 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:32.699068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:32.701260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ffa93-7019-47d8-980f-91769bdfbc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:32.701938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ffa93-7019-47d8-980f-91769bdfbc64 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:32.702222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ffa93-7019-47d8-980f-91769bdfbc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:32.702599 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ffa93-7019-47d8-980f-91769bdfbc64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:32.703309 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ffa93-7019-47d8-980f-91769bdfbc64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:32.705262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ffa93-7019-47d8-980f-91769bdfbc64 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:32.775528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1349/2696] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:51:32 2022] POST /identity/v3/projects => generated 389 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:51:32.780886 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ffa93-7019-47d8-980f-91769bdfbc64 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:32.843204 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315694d3-2742-4cc0-b04c-99cfebeec93f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:32.951810 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ce21f-5cb1-4a18-85c0-6ba426e81977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:32.952802 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ce21f-5cb1-4a18-85c0-6ba426e81977 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:32.952802 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ce21f-5cb1-4a18-85c0-6ba426e81977 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:32.954903 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ce21f-5cb1-4a18-85c0-6ba426e81977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:32.958542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f97ce21f-5cb1-4a18-85c0-6ba426e81977 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:32.967241 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ffa93-7019-47d8-980f-91769bdfbc64 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:32.978781 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453323b-de02-40de-81c4-d3ab53c8b985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:32.980509 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453323b-de02-40de-81c4-d3ab53c8b985 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:32.982257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453323b-de02-40de-81c4-d3ab53c8b985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:32.983317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453323b-de02-40de-81c4-d3ab53c8b985 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:32.985902 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ce21f-5cb1-4a18-85c0-6ba426e81977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:32.988170 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453323b-de02-40de-81c4-d3ab53c8b985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:32.991066 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ce21f-5cb1-4a18-85c0-6ba426e81977 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:32.997204 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453323b-de02-40de-81c4-d3ab53c8b985 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:33.013843 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1348/2697] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:51:33.038860 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1350/2698] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:51:32 2022] POST /identity/v3/users => generated 295 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:51:33.041153 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ce21f-5cb1-4a18-85c0-6ba426e81977 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:33.052762 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ce21f-5cb1-4a18-85c0-6ba426e81977 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:33.062992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb87c55-e382-4a25-bb9f-8ee5d6b95b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:33.064926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb87c55-e382-4a25-bb9f-8ee5d6b95b16 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:33.065214 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb87c55-e382-4a25-bb9f-8ee5d6b95b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:33.065670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb87c55-e382-4a25-bb9f-8ee5d6b95b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:33.066274 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb87c55-e382-4a25-bb9f-8ee5d6b95b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:33.068588 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb87c55-e382-4a25-bb9f-8ee5d6b95b16 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:33.080904 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1349/2699] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:51:33.085260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453323b-de02-40de-81c4-d3ab53c8b985 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:33.097744 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453323b-de02-40de-81c4-d3ab53c8b985 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:33.105117 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c874b-d0ef-406e-acd7-ec98817544a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:33.106101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c874b-d0ef-406e-acd7-ec98817544a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:33.106571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c874b-d0ef-406e-acd7-ec98817544a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:33.107011 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c874b-d0ef-406e-acd7-ec98817544a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:33.107923 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c874b-d0ef-406e-acd7-ec98817544a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:33.110056 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c874b-d0ef-406e-acd7-ec98817544a9 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:33.130115 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb87c55-e382-4a25-bb9f-8ee5d6b95b16 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:33.138349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1351/2700] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:51:33.143896 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb87c55-e382-4a25-bb9f-8ee5d6b95b16 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:33.151706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97546e8-52e6-4470-a2d3-79fad9845a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:33.152950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97546e8-52e6-4470-a2d3-79fad9845a10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:33.153322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97546e8-52e6-4470-a2d3-79fad9845a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:33.153768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97546e8-52e6-4470-a2d3-79fad9845a10 admin admin] PATH_INFO: `/v3/projects/a1a3251710054d589ceeee91cf556bbc/users/09c401e432344e0eb2204111e48c3730/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:33.177792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97546e8-52e6-4470-a2d3-79fad9845a10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a3251710054d589ceeee91cf556bbc, user_id=09c401e432344e0eb2204111e48c3730, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:33.181888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97546e8-52e6-4470-a2d3-79fad9845a10 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:33.206577 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1350/2701] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:51:33 2022] PUT /identity/v3/projects/a1a3251710054d589ceeee91cf556bbc/users/09c401e432344e0eb2204111e48c3730/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:33.218694 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcfcf7-682a-43c0-a85b-2e17d94e9af9 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:33.219122 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcfcf7-682a-43c0-a85b-2e17d94e9af9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:33.219547 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcfcf7-682a-43c0-a85b-2e17d94e9af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:33.259674 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e1bcfcf7-682a-43c0-a85b-2e17d94e9af9 None None] MFA Rules not processed for user `09c401e432344e0eb2204111e48c3730`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:33.261720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e1bcfcf7-682a-43c0-a85b-2e17d94e9af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:33.352747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:33.387337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1352/2702] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:33.407699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97546e8-52e6-4470-a2d3-79fad9845a10 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:33.411699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f97546e8-52e6-4470-a2d3-79fad9845a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:33.560515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97546e8-52e6-4470-a2d3-79fad9845a10 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:33.573159 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bed09-3a3e-4610-a74b-4a0482508d9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:33.573943 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bed09-3a3e-4610-a74b-4a0482508d9a service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:33.575596 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bed09-3a3e-4610-a74b-4a0482508d9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:33.576863 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bed09-3a3e-4610-a74b-4a0482508d9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:33.602167 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bed09-3a3e-4610-a74b-4a0482508d9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:33.605061 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bed09-3a3e-4610-a74b-4a0482508d9a service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:33.624192 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1351/2703] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:36.491447 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcfcf7-682a-43c0-a85b-2e17d94e9af9 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:36.495305 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e1bcfcf7-682a-43c0-a85b-2e17d94e9af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:36.664512 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcfcf7-682a-43c0-a85b-2e17d94e9af9 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:36.687669 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35743185-fc35-49c9-ae8c-d226f64f871e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:36.688829 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35743185-fc35-49c9-ae8c-d226f64f871e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:36.689580 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35743185-fc35-49c9-ae8c-d226f64f871e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:36.689889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35743185-fc35-49c9-ae8c-d226f64f871e admin admin] PATH_INFO: `/v3/users/7cb192800160427e9951a759dd10b228` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:36.694562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35743185-fc35-49c9-ae8c-d226f64f871e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cb192800160427e9951a759dd10b228)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:36.697165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35743185-fc35-49c9-ae8c-d226f64f871e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:36.809773 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-35743185-fc35-49c9-ae8c-d226f64f871e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cb192800160427e9951a759dd10b228'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:36.824431 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-35743185-fc35-49c9-ae8c-d226f64f871e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cb192800160427e9951a759dd10b228'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:36.844263 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-35743185-fc35-49c9-ae8c-d226f64f871e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cb192800160427e9951a759dd10b228'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:36.845381 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-35743185-fc35-49c9-ae8c-d226f64f871e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cb192800160427e9951a759dd10b228'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:36.846390 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-35743185-fc35-49c9-ae8c-d226f64f871e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cb192800160427e9951a759dd10b228'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:36.917044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1353/2704] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:36 2022] DELETE /identity/v3/users/7cb192800160427e9951a759dd10b228 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:37.277343 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bed09-3a3e-4610-a74b-4a0482508d9a service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:37.295800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bed09-3a3e-4610-a74b-4a0482508d9a service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:37.417895 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:37.460586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34373c3-a017-48ab-82a5-562f68c47699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:37.462707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34373c3-a017-48ab-82a5-562f68c47699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:37.463310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34373c3-a017-48ab-82a5-562f68c47699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:37.473018 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34373c3-a017-48ab-82a5-562f68c47699 admin admin] PATH_INFO: `/v3/projects/8c2e98a33cba4c08a55cda8dd6ed0730` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:37.478626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34373c3-a017-48ab-82a5-562f68c47699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c2e98a33cba4c08a55cda8dd6ed0730)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:37.482052 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34373c3-a017-48ab-82a5-562f68c47699 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:37.594746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e34373c3-a017-48ab-82a5-562f68c47699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c2e98a33cba4c08a55cda8dd6ed0730', 'request_id': 'req-e34373c3-a017-48ab-82a5-562f68c47699'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:37.607102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e34373c3-a017-48ab-82a5-562f68c47699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c2e98a33cba4c08a55cda8dd6ed0730', 'request_id': 'req-e34373c3-a017-48ab-82a5-562f68c47699'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:37.618488 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e34373c3-a017-48ab-82a5-562f68c47699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c2e98a33cba4c08a55cda8dd6ed0730', 'request_id': 'req-e34373c3-a017-48ab-82a5-562f68c47699'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:37.639147 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e34373c3-a017-48ab-82a5-562f68c47699 admin admin] The token cache is being invalidate because project 8c2e98a33cba4c08a55cda8dd6ed0730 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:51:37.639594 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e34373c3-a017-48ab-82a5-562f68c47699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:37.640534 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e34373c3-a017-48ab-82a5-562f68c47699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:37.642312 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1352/2705] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:51:37 2022] DELETE /identity/v3/projects/8c2e98a33cba4c08a55cda8dd6ed0730 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:37.647439 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35743185-fc35-49c9-ae8c-d226f64f871e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:37.651349 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-35743185-fc35-49c9-ae8c-d226f64f871e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:37.670367 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35743185-fc35-49c9-ae8c-d226f64f871e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:37.756525 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece3367-d78b-4ce9-be8b-aa5b4d18239c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:37.756771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece3367-d78b-4ce9-be8b-aa5b4d18239c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:37.757108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece3367-d78b-4ce9-be8b-aa5b4d18239c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:37.757396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece3367-d78b-4ce9-be8b-aa5b4d18239c admin admin] PATH_INFO: `/v3/users/987a8f8932774c3ab348360a03cf5eb8` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:37.759359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece3367-d78b-4ce9-be8b-aa5b4d18239c admin admin] RBAC: Authorizing `identity:delete_user(user_id=987a8f8932774c3ab348360a03cf5eb8)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:37.762290 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece3367-d78b-4ce9-be8b-aa5b4d18239c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:37.788695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:37.858493 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3ece3367-d78b-4ce9-be8b-aa5b4d18239c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '987a8f8932774c3ab348360a03cf5eb8'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:37.871951 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3ece3367-d78b-4ce9-be8b-aa5b4d18239c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '987a8f8932774c3ab348360a03cf5eb8'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:37.883771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3ece3367-d78b-4ce9-be8b-aa5b4d18239c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '987a8f8932774c3ab348360a03cf5eb8'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:37.884444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3ece3367-d78b-4ce9-be8b-aa5b4d18239c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '987a8f8932774c3ab348360a03cf5eb8'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:37.885079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3ece3367-d78b-4ce9-be8b-aa5b4d18239c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '987a8f8932774c3ab348360a03cf5eb8'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:37.915331 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1354/2706] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:37 2022] DELETE /identity/v3/users/987a8f8932774c3ab348360a03cf5eb8 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:38.319211 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34373c3-a017-48ab-82a5-562f68c47699 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:38.322792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e34373c3-a017-48ab-82a5-562f68c47699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:38.373189 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34373c3-a017-48ab-82a5-562f68c47699 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:38.533980 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466fc9e-4b65-48e8-badb-c6cda90a6a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:38.535255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466fc9e-4b65-48e8-badb-c6cda90a6a90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:38.537056 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466fc9e-4b65-48e8-badb-c6cda90a6a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:38.537619 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466fc9e-4b65-48e8-badb-c6cda90a6a90 admin admin] PATH_INFO: `/v3/projects/e0b0600de07d4f01aaffe1b92d31aa49` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:38.540175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466fc9e-4b65-48e8-badb-c6cda90a6a90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0b0600de07d4f01aaffe1b92d31aa49)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:38.544170 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466fc9e-4b65-48e8-badb-c6cda90a6a90 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:38.718177 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0466fc9e-4b65-48e8-badb-c6cda90a6a90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0b0600de07d4f01aaffe1b92d31aa49', 'request_id': 'req-0466fc9e-4b65-48e8-badb-c6cda90a6a90'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:38.747553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0466fc9e-4b65-48e8-badb-c6cda90a6a90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0b0600de07d4f01aaffe1b92d31aa49', 'request_id': 'req-0466fc9e-4b65-48e8-badb-c6cda90a6a90'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:38.796089 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0466fc9e-4b65-48e8-badb-c6cda90a6a90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0b0600de07d4f01aaffe1b92d31aa49', 'request_id': 'req-0466fc9e-4b65-48e8-badb-c6cda90a6a90'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:38.811279 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:38.816418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0466fc9e-4b65-48e8-badb-c6cda90a6a90 admin admin] The token cache is being invalidate because project e0b0600de07d4f01aaffe1b92d31aa49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:51:38.817051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0466fc9e-4b65-48e8-badb-c6cda90a6a90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:38.818337 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0466fc9e-4b65-48e8-badb-c6cda90a6a90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:38.820654 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1353/2707] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:51:38 2022] DELETE /identity/v3/projects/e0b0600de07d4f01aaffe1b92d31aa49 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:39.345556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece3367-d78b-4ce9-be8b-aa5b4d18239c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:39.347008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3ece3367-d78b-4ce9-be8b-aa5b4d18239c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:39.373867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece3367-d78b-4ce9-be8b-aa5b4d18239c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:39.472853 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce3661-bdf6-414a-90a7-7d17afd23eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:39.474160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce3661-bdf6-414a-90a7-7d17afd23eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:39.475098 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce3661-bdf6-414a-90a7-7d17afd23eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:39.477080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce3661-bdf6-414a-90a7-7d17afd23eff admin admin] PATH_INFO: `/v3/users/5744cccc5a224263b71c052dc783e103` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:39.481985 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce3661-bdf6-414a-90a7-7d17afd23eff admin admin] RBAC: Authorizing `identity:delete_user(user_id=5744cccc5a224263b71c052dc783e103)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:39.485820 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce3661-bdf6-414a-90a7-7d17afd23eff admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:39.594988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e7ce3661-bdf6-414a-90a7-7d17afd23eff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5744cccc5a224263b71c052dc783e103'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:39.618329 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e7ce3661-bdf6-414a-90a7-7d17afd23eff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5744cccc5a224263b71c052dc783e103'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:39.641178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e7ce3661-bdf6-414a-90a7-7d17afd23eff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5744cccc5a224263b71c052dc783e103'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:39.643472 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e7ce3661-bdf6-414a-90a7-7d17afd23eff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5744cccc5a224263b71c052dc783e103'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:39.645101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e7ce3661-bdf6-414a-90a7-7d17afd23eff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5744cccc5a224263b71c052dc783e103'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:39.681698 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1355/2708] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:39 2022] DELETE /identity/v3/users/5744cccc5a224263b71c052dc783e103 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:51:39.838001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466fc9e-4b65-48e8-badb-c6cda90a6a90 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:39.839837 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0466fc9e-4b65-48e8-badb-c6cda90a6a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:39.911040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466fc9e-4b65-48e8-badb-c6cda90a6a90 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:39.928480 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce3661-bdf6-414a-90a7-7d17afd23eff admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:39.930930 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e7ce3661-bdf6-414a-90a7-7d17afd23eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:39.965409 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:39.993421 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8ad86-543d-4cfb-a640-6e6915082a0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:39.994091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8ad86-543d-4cfb-a640-6e6915082a0f service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:39.994371 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8ad86-543d-4cfb-a640-6e6915082a0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:39.994642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8ad86-543d-4cfb-a640-6e6915082a0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:39.996753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-36b8ad86-543d-4cfb-a640-6e6915082a0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:40.094541 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce3661-bdf6-414a-90a7-7d17afd23eff admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:40.101502 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aab3d2-a63b-4d1c-a6dc-c153845de60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:40.102335 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aab3d2-a63b-4d1c-a6dc-c153845de60f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:40.102852 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aab3d2-a63b-4d1c-a6dc-c153845de60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:40.103247 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aab3d2-a63b-4d1c-a6dc-c153845de60f admin admin] PATH_INFO: `/v3/projects/157914bad58e4c50b92dfca14ec638aa` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:40.105282 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aab3d2-a63b-4d1c-a6dc-c153845de60f admin admin] RBAC: Authorizing `identity:delete_project(project_id=157914bad58e4c50b92dfca14ec638aa)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:40.107685 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aab3d2-a63b-4d1c-a6dc-c153845de60f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:40.109888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8ad86-543d-4cfb-a640-6e6915082a0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:40.112823 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8ad86-543d-4cfb-a640-6e6915082a0f service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:40.135938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1354/2709] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:40.184005 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e7aab3d2-a63b-4d1c-a6dc-c153845de60f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '157914bad58e4c50b92dfca14ec638aa', 'request_id': 'req-e7aab3d2-a63b-4d1c-a6dc-c153845de60f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:40.195971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e7aab3d2-a63b-4d1c-a6dc-c153845de60f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '157914bad58e4c50b92dfca14ec638aa', 'request_id': 'req-e7aab3d2-a63b-4d1c-a6dc-c153845de60f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:40.208209 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e7aab3d2-a63b-4d1c-a6dc-c153845de60f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '157914bad58e4c50b92dfca14ec638aa', 'request_id': 'req-e7aab3d2-a63b-4d1c-a6dc-c153845de60f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:40.221727 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e7aab3d2-a63b-4d1c-a6dc-c153845de60f admin admin] The token cache is being invalidate because project 157914bad58e4c50b92dfca14ec638aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:51:40.221727 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e7aab3d2-a63b-4d1c-a6dc-c153845de60f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:40.222397 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e7aab3d2-a63b-4d1c-a6dc-c153845de60f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:40.224259 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1356/2710] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:51:39 2022] DELETE /identity/v3/projects/157914bad58e4c50b92dfca14ec638aa => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:40.229616 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8387c0-2b65-4d9f-9dc5-872c244b7ec0 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:40.229979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8387c0-2b65-4d9f-9dc5-872c244b7ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:40.230385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8387c0-2b65-4d9f-9dc5-872c244b7ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:40.256296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-db8387c0-2b65-4d9f-9dc5-872c244b7ec0 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:40.258251 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-db8387c0-2b65-4d9f-9dc5-872c244b7ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:40.332673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1355/2711] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:40.341157 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9dc5e-ddca-4a38-a783-c27401a4270d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:40.341438 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9dc5e-ddca-4a38-a783-c27401a4270d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:40.341762 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9dc5e-ddca-4a38-a783-c27401a4270d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:40.364201 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-13f9dc5e-ddca-4a38-a783-c27401a4270d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:40.365937 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-13f9dc5e-ddca-4a38-a783-c27401a4270d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:40.377195 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1357/2712] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:40.382327 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8387c0-2b65-4d9f-9dc5-872c244b7ec0 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:40.393842 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8387c0-2b65-4d9f-9dc5-872c244b7ec0 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:40.399130 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4edfda-a3f2-4612-8041-c512791d2103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:40.399788 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4edfda-a3f2-4612-8041-c512791d2103 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:40.400102 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4edfda-a3f2-4612-8041-c512791d2103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:40.400380 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4edfda-a3f2-4612-8041-c512791d2103 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:40.401103 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4edfda-a3f2-4612-8041-c512791d2103 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:51:40.401518 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4edfda-a3f2-4612-8041-c512791d2103 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:40.403137 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4edfda-a3f2-4612-8041-c512791d2103 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:40.404926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:40.418048 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1356/2713] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:51:40.422984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9dc5e-ddca-4a38-a783-c27401a4270d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:40.441524 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9dc5e-ddca-4a38-a783-c27401a4270d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:40.446895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb77c4-3980-4376-aed6-40b6db650644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:40.447558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb77c4-3980-4376-aed6-40b6db650644 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:40.447947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb77c4-3980-4376-aed6-40b6db650644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:40.448103 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb77c4-3980-4376-aed6-40b6db650644 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:40.448902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb77c4-3980-4376-aed6-40b6db650644 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:40.451229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb77c4-3980-4376-aed6-40b6db650644 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:40.496773 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1358/2714] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:51:40 2022] POST /identity/v3/projects => generated 403 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:51:40.502659 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4edfda-a3f2-4612-8041-c512791d2103 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:40.514164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4edfda-a3f2-4612-8041-c512791d2103 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:40.599844 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959bbcb-adc4-4ae8-8fce-0faa96b1f8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:40.600500 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959bbcb-adc4-4ae8-8fce-0faa96b1f8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:40.600781 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959bbcb-adc4-4ae8-8fce-0faa96b1f8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:40.601057 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959bbcb-adc4-4ae8-8fce-0faa96b1f8c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:40.601982 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959bbcb-adc4-4ae8-8fce-0faa96b1f8c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:40.604381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959bbcb-adc4-4ae8-8fce-0faa96b1f8c7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:40.638544 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1357/2715] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:51:40 2022] POST /identity/v3/users => generated 302 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:51:40.643636 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb77c4-3980-4376-aed6-40b6db650644 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:40.666290 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb77c4-3980-4376-aed6-40b6db650644 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:40.678080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a1d8e-1749-4f08-830f-c8d7833889e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:40.678944 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a1d8e-1749-4f08-830f-c8d7833889e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:40.685684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a1d8e-1749-4f08-830f-c8d7833889e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:40.685684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a1d8e-1749-4f08-830f-c8d7833889e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:40.685684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a1d8e-1749-4f08-830f-c8d7833889e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:40.687642 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a1d8e-1749-4f08-830f-c8d7833889e9 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:40.709528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1359/2716] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:51:40.715440 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959bbcb-adc4-4ae8-8fce-0faa96b1f8c7 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:40.729856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959bbcb-adc4-4ae8-8fce-0faa96b1f8c7 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:40.736499 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12c775-4ed0-47a3-8d4f-9f61f71af6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:40.737306 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12c775-4ed0-47a3-8d4f-9f61f71af6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:40.737882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12c775-4ed0-47a3-8d4f-9f61f71af6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:40.738289 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12c775-4ed0-47a3-8d4f-9f61f71af6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:40.739514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12c775-4ed0-47a3-8d4f-9f61f71af6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:40.742587 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12c775-4ed0-47a3-8d4f-9f61f71af6b7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:40.756679 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1358/2717] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:51:40.762482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a1d8e-1749-4f08-830f-c8d7833889e9 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:40.775338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a1d8e-1749-4f08-830f-c8d7833889e9 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:40.780718 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c7ed8-896e-4d2d-83b6-185c14d38e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:40.781482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c7ed8-896e-4d2d-83b6-185c14d38e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:40.781783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c7ed8-896e-4d2d-83b6-185c14d38e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:40.782070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c7ed8-896e-4d2d-83b6-185c14d38e6f admin admin] PATH_INFO: `/v3/projects/fc679cb81aad416b8007d02bd01e31d4/users/a2c059adc1c04069b413995e1e537279/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:40.799385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c7ed8-896e-4d2d-83b6-185c14d38e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc679cb81aad416b8007d02bd01e31d4, user_id=a2c059adc1c04069b413995e1e537279, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:40.802175 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c7ed8-896e-4d2d-83b6-185c14d38e6f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:40.819551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1360/2718] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:51:40 2022] PUT /identity/v3/projects/fc679cb81aad416b8007d02bd01e31d4/users/a2c059adc1c04069b413995e1e537279/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:40.831205 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a678070-3f0c-4bb6-ae76-2d2c27204f4e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:40.831558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a678070-3f0c-4bb6-ae76-2d2c27204f4e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:40.831764 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a678070-3f0c-4bb6-ae76-2d2c27204f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:40.865403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3a678070-3f0c-4bb6-ae76-2d2c27204f4e None None] MFA Rules not processed for user `a2c059adc1c04069b413995e1e537279`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:40.867730 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3a678070-3f0c-4bb6-ae76-2d2c27204f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:40.963231 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1359/2719] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:40 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:40.969224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c7ed8-896e-4d2d-83b6-185c14d38e6f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:41.031756 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:41.069173 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c7ed8-896e-4d2d-83b6-185c14d38e6f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:41.077091 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1967d2-e92c-472d-9c4a-7060695a0c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:41.078179 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1967d2-e92c-472d-9c4a-7060695a0c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:41.078697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1967d2-e92c-472d-9c4a-7060695a0c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:41.079000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1967d2-e92c-472d-9c4a-7060695a0c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:41.079785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1967d2-e92c-472d-9c4a-7060695a0c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:41.083022 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1967d2-e92c-472d-9c4a-7060695a0c2f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:41.130967 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1361/2720] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:51:40 2022] POST /identity/v3/projects => generated 403 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:51:41.134697 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a678070-3f0c-4bb6-ae76-2d2c27204f4e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:41.164889 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a678070-3f0c-4bb6-ae76-2d2c27204f4e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:41.251140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42961de-0a29-436b-9a30-969ea178d36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:41.251787 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42961de-0a29-436b-9a30-969ea178d36c admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:41.252073 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42961de-0a29-436b-9a30-969ea178d36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:41.252360 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42961de-0a29-436b-9a30-969ea178d36c admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:41.253075 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42961de-0a29-436b-9a30-969ea178d36c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:41.255091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42961de-0a29-436b-9a30-969ea178d36c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:41.266407 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:41.286285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1360/2721] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:51:41 2022] POST /identity/v3/users => generated 308 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:51:41.290979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1967d2-e92c-472d-9c4a-7060695a0c2f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:41.306224 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1967d2-e92c-472d-9c4a-7060695a0c2f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:41.314215 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca56a42-530f-4a40-9b84-68b150ce78ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:41.315311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca56a42-530f-4a40-9b84-68b150ce78ad admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:41.315835 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca56a42-530f-4a40-9b84-68b150ce78ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:41.316329 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca56a42-530f-4a40-9b84-68b150ce78ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:41.317299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca56a42-530f-4a40-9b84-68b150ce78ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:41.320969 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca56a42-530f-4a40-9b84-68b150ce78ad admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:41.345342 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1362/2722] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:51:41.346784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42961de-0a29-436b-9a30-969ea178d36c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:41.357385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42961de-0a29-436b-9a30-969ea178d36c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:41.362977 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088e1a5-4ac1-4bd2-a0cd-7327efbbd1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:41.363694 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088e1a5-4ac1-4bd2-a0cd-7327efbbd1e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:41.364055 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088e1a5-4ac1-4bd2-a0cd-7327efbbd1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:41.364387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088e1a5-4ac1-4bd2-a0cd-7327efbbd1e5 admin admin] PATH_INFO: `/v3/projects/70ec33849de3457a8b04b2f20953e999/users/9eb9136b18454b388e2efbcf659df4cf/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:41.381920 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088e1a5-4ac1-4bd2-a0cd-7327efbbd1e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70ec33849de3457a8b04b2f20953e999, user_id=9eb9136b18454b388e2efbcf659df4cf, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:41.386108 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088e1a5-4ac1-4bd2-a0cd-7327efbbd1e5 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:41.402867 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1361/2723] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:51:41 2022] PUT /identity/v3/projects/70ec33849de3457a8b04b2f20953e999/users/9eb9136b18454b388e2efbcf659df4cf/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:51:41.408302 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca56a42-530f-4a40-9b84-68b150ce78ad admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:41.425754 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca56a42-530f-4a40-9b84-68b150ce78ad admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:41.498889 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbab77b-7756-441d-9eb3-0a870b04d153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:41.499532 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbab77b-7756-441d-9eb3-0a870b04d153 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:41.499813 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbab77b-7756-441d-9eb3-0a870b04d153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:41.500089 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbab77b-7756-441d-9eb3-0a870b04d153 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:41.500861 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbab77b-7756-441d-9eb3-0a870b04d153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:41.502564 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbab77b-7756-441d-9eb3-0a870b04d153 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:41.520284 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1363/2724] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:51:41.521858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088e1a5-4ac1-4bd2-a0cd-7327efbbd1e5 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:41.532878 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088e1a5-4ac1-4bd2-a0cd-7327efbbd1e5 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:41.537719 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf0011-68a2-4048-8f37-710e06bea0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:41.538426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf0011-68a2-4048-8f37-710e06bea0e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:41.538714 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf0011-68a2-4048-8f37-710e06bea0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:41.539040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf0011-68a2-4048-8f37-710e06bea0e6 admin admin] PATH_INFO: `/v3/projects/70ec33849de3457a8b04b2f20953e999/users/9eb9136b18454b388e2efbcf659df4cf/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:41.541132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf0011-68a2-4048-8f37-710e06bea0e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70ec33849de3457a8b04b2f20953e999, user_id=9eb9136b18454b388e2efbcf659df4cf, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:41.543870 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf0011-68a2-4048-8f37-710e06bea0e6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:41.557125 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1362/2725] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:51:41 2022] PUT /identity/v3/projects/70ec33849de3457a8b04b2f20953e999/users/9eb9136b18454b388e2efbcf659df4cf/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:51:41.569334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e0c9a-8bfa-402c-9fd6-8315bc357418 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:41.569673 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e0c9a-8bfa-402c-9fd6-8315bc357418 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:41.569959 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e0c9a-8bfa-402c-9fd6-8315bc357418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:41.601199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-507e0c9a-8bfa-402c-9fd6-8315bc357418 None None] MFA Rules not processed for user `9eb9136b18454b388e2efbcf659df4cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:41.602494 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-507e0c9a-8bfa-402c-9fd6-8315bc357418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:41.723115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1364/2726] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:41 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:41.737069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf0011-68a2-4048-8f37-710e06bea0e6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:41.739425 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-02bf0011-68a2-4048-8f37-710e06bea0e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:41.854324 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf0011-68a2-4048-8f37-710e06bea0e6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:41.859616 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec268a6b-8f2b-43db-8d3e-45b6b581a7c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:41.860388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec268a6b-8f2b-43db-8d3e-45b6b581a7c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:41.860864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec268a6b-8f2b-43db-8d3e-45b6b581a7c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:41.861223 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec268a6b-8f2b-43db-8d3e-45b6b581a7c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:41.874591 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec268a6b-8f2b-43db-8d3e-45b6b581a7c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:41.876464 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec268a6b-8f2b-43db-8d3e-45b6b581a7c9 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:41.891405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1363/2727] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:41 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:42.080337 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e0c9a-8bfa-402c-9fd6-8315bc357418 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:42.134662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:42.178117 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e0c9a-8bfa-402c-9fd6-8315bc357418 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:42.183656 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad76adc-401e-49fa-91af-fed1146ae7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:42.184343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad76adc-401e-49fa-91af-fed1146ae7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:42.184621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad76adc-401e-49fa-91af-fed1146ae7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:42.184900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad76adc-401e-49fa-91af-fed1146ae7c8 admin admin] PATH_INFO: `/v3/users/a2c059adc1c04069b413995e1e537279` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:42.186341 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad76adc-401e-49fa-91af-fed1146ae7c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2c059adc1c04069b413995e1e537279)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:42.189689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad76adc-401e-49fa-91af-fed1146ae7c8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:42.262588 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5ad76adc-401e-49fa-91af-fed1146ae7c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2c059adc1c04069b413995e1e537279'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:42.271723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5ad76adc-401e-49fa-91af-fed1146ae7c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2c059adc1c04069b413995e1e537279'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:42.280891 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5ad76adc-401e-49fa-91af-fed1146ae7c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2c059adc1c04069b413995e1e537279'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:42.281740 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5ad76adc-401e-49fa-91af-fed1146ae7c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2c059adc1c04069b413995e1e537279'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:42.282593 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5ad76adc-401e-49fa-91af-fed1146ae7c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2c059adc1c04069b413995e1e537279'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:42.307550 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1365/2728] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:42 2022] DELETE /identity/v3/users/a2c059adc1c04069b413995e1e537279 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:42.313403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b094f8f-4a52-4c30-8513-62b2af974425 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:42.313839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b094f8f-4a52-4c30-8513-62b2af974425 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:42.314188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b094f8f-4a52-4c30-8513-62b2af974425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:42.336853 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5b094f8f-4a52-4c30-8513-62b2af974425 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:42.338603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5b094f8f-4a52-4c30-8513-62b2af974425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:42.430929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1364/2729] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:42.453285 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad76adc-401e-49fa-91af-fed1146ae7c8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:42.456634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5ad76adc-401e-49fa-91af-fed1146ae7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:42.559257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad76adc-401e-49fa-91af-fed1146ae7c8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:42.564016 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751752a0-fcaa-46c1-b55b-543258ba3a5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:42.564888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751752a0-fcaa-46c1-b55b-543258ba3a5e service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:42.565716 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751752a0-fcaa-46c1-b55b-543258ba3a5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:42.565716 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751752a0-fcaa-46c1-b55b-543258ba3a5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:42.578953 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b094f8f-4a52-4c30-8513-62b2af974425 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:42.579205 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751752a0-fcaa-46c1-b55b-543258ba3a5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:42.580834 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5b094f8f-4a52-4c30-8513-62b2af974425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:42.581227 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751752a0-fcaa-46c1-b55b-543258ba3a5e service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:42.597317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1366/2730] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:42.768295 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:42.838227 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b094f8f-4a52-4c30-8513-62b2af974425 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:42.857278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69913f-3e58-43cd-b5d8-b2454239551f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:42.858279 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69913f-3e58-43cd-b5d8-b2454239551f service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:42.858929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69913f-3e58-43cd-b5d8-b2454239551f service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:42.859253 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69913f-3e58-43cd-b5d8-b2454239551f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:42.862078 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0f69913f-3e58-43cd-b5d8-b2454239551f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:42.886530 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69913f-3e58-43cd-b5d8-b2454239551f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:42.888420 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69913f-3e58-43cd-b5d8-b2454239551f service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:42.906423 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1365/2731] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:43.015161 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751752a0-fcaa-46c1-b55b-543258ba3a5e service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:43.017503 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-751752a0-fcaa-46c1-b55b-543258ba3a5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:43.059915 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751752a0-fcaa-46c1-b55b-543258ba3a5e service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:43.070953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415db89-80f0-4795-b2ee-2a324b4c5a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:43.071735 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415db89-80f0-4795-b2ee-2a324b4c5a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:43.072404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415db89-80f0-4795-b2ee-2a324b4c5a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:43.072822 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415db89-80f0-4795-b2ee-2a324b4c5a1a admin admin] PATH_INFO: `/v3/projects/fc679cb81aad416b8007d02bd01e31d4` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:43.074706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415db89-80f0-4795-b2ee-2a324b4c5a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc679cb81aad416b8007d02bd01e31d4)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:43.076273 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415db89-80f0-4795-b2ee-2a324b4c5a1a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:43.339067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:43.343108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e415db89-80f0-4795-b2ee-2a324b4c5a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc679cb81aad416b8007d02bd01e31d4', 'request_id': 'req-e415db89-80f0-4795-b2ee-2a324b4c5a1a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:43.362430 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e415db89-80f0-4795-b2ee-2a324b4c5a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc679cb81aad416b8007d02bd01e31d4', 'request_id': 'req-e415db89-80f0-4795-b2ee-2a324b4c5a1a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:43.379696 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e415db89-80f0-4795-b2ee-2a324b4c5a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc679cb81aad416b8007d02bd01e31d4', 'request_id': 'req-e415db89-80f0-4795-b2ee-2a324b4c5a1a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:43.413257 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e415db89-80f0-4795-b2ee-2a324b4c5a1a admin admin] The token cache is being invalidate because project fc679cb81aad416b8007d02bd01e31d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:51:43.413699 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e415db89-80f0-4795-b2ee-2a324b4c5a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:43.415191 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e415db89-80f0-4795-b2ee-2a324b4c5a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:43.416881 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1367/2732] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:51:43 2022] DELETE /identity/v3/projects/fc679cb81aad416b8007d02bd01e31d4 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:43.422332 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69913f-3e58-43cd-b5d8-b2454239551f service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:43.438581 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69913f-3e58-43cd-b5d8-b2454239551f service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:43.579550 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f3608-16ef-4aea-9466-f53039efc402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:43.580531 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f3608-16ef-4aea-9466-f53039efc402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:43.580978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f3608-16ef-4aea-9466-f53039efc402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:43.581443 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f3608-16ef-4aea-9466-f53039efc402 admin admin] PATH_INFO: `/v3/users/9eb9136b18454b388e2efbcf659df4cf` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:43.583765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f3608-16ef-4aea-9466-f53039efc402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eb9136b18454b388e2efbcf659df4cf)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:43.588296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f3608-16ef-4aea-9466-f53039efc402 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:43.686474 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-433f3608-16ef-4aea-9466-f53039efc402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eb9136b18454b388e2efbcf659df4cf'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:43.718745 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-433f3608-16ef-4aea-9466-f53039efc402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eb9136b18454b388e2efbcf659df4cf'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:43.719900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-433f3608-16ef-4aea-9466-f53039efc402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eb9136b18454b388e2efbcf659df4cf'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:43.734767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-433f3608-16ef-4aea-9466-f53039efc402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eb9136b18454b388e2efbcf659df4cf'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:43.752418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-433f3608-16ef-4aea-9466-f53039efc402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eb9136b18454b388e2efbcf659df4cf'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:43.765512 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1366/2733] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:43 2022] DELETE /identity/v3/users/9eb9136b18454b388e2efbcf659df4cf => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:44.308943 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415db89-80f0-4795-b2ee-2a324b4c5a1a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:44.310738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e415db89-80f0-4795-b2ee-2a324b4c5a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:44.326971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415db89-80f0-4795-b2ee-2a324b4c5a1a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:44.451742 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34098f-4fde-42c1-a4e3-c7e6b5cb3c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:44.453008 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34098f-4fde-42c1-a4e3-c7e6b5cb3c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:44.453304 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34098f-4fde-42c1-a4e3-c7e6b5cb3c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:44.453814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34098f-4fde-42c1-a4e3-c7e6b5cb3c12 admin admin] PATH_INFO: `/v3/projects/70ec33849de3457a8b04b2f20953e999` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:44.455719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34098f-4fde-42c1-a4e3-c7e6b5cb3c12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70ec33849de3457a8b04b2f20953e999)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:44.458624 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34098f-4fde-42c1-a4e3-c7e6b5cb3c12 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:44.537681 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2d34098f-4fde-42c1-a4e3-c7e6b5cb3c12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70ec33849de3457a8b04b2f20953e999', 'request_id': 'req-2d34098f-4fde-42c1-a4e3-c7e6b5cb3c12'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:44.545706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2d34098f-4fde-42c1-a4e3-c7e6b5cb3c12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70ec33849de3457a8b04b2f20953e999', 'request_id': 'req-2d34098f-4fde-42c1-a4e3-c7e6b5cb3c12'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:44.555933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2d34098f-4fde-42c1-a4e3-c7e6b5cb3c12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70ec33849de3457a8b04b2f20953e999', 'request_id': 'req-2d34098f-4fde-42c1-a4e3-c7e6b5cb3c12'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:44.582039 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2d34098f-4fde-42c1-a4e3-c7e6b5cb3c12 admin admin] The token cache is being invalidate because project 70ec33849de3457a8b04b2f20953e999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:51:44.582523 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2d34098f-4fde-42c1-a4e3-c7e6b5cb3c12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:44.583614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2d34098f-4fde-42c1-a4e3-c7e6b5cb3c12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:44.586643 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1368/2734] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:51:44 2022] DELETE /identity/v3/projects/70ec33849de3457a8b04b2f20953e999 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:44.594809 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e338141-151e-4a2c-92e0-1628204db2b1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:44.595637 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e338141-151e-4a2c-92e0-1628204db2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:44.595804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e338141-151e-4a2c-92e0-1628204db2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:44.629085 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2e338141-151e-4a2c-92e0-1628204db2b1 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:44.631240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2e338141-151e-4a2c-92e0-1628204db2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:44.697046 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:44.746862 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1367/2735] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:44.755177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efecfd8-9220-4934-b102-d5bf7eb3b6b9 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:44.755476 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efecfd8-9220-4934-b102-d5bf7eb3b6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:44.755717 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efecfd8-9220-4934-b102-d5bf7eb3b6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:44.787362 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3efecfd8-9220-4934-b102-d5bf7eb3b6b9 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:44.790153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3efecfd8-9220-4934-b102-d5bf7eb3b6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:44.804198 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1369/2736] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:44.809280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e338141-151e-4a2c-92e0-1628204db2b1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:44.827515 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e338141-151e-4a2c-92e0-1628204db2b1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:44.836154 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3752c3-18b3-46a8-8ae5-2a7cb21f9010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:44.837251 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3752c3-18b3-46a8-8ae5-2a7cb21f9010 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:44.837792 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3752c3-18b3-46a8-8ae5-2a7cb21f9010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:44.838272 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3752c3-18b3-46a8-8ae5-2a7cb21f9010 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:44.839766 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3752c3-18b3-46a8-8ae5-2a7cb21f9010 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:51:44.840319 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3752c3-18b3-46a8-8ae5-2a7cb21f9010 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:44.843417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3752c3-18b3-46a8-8ae5-2a7cb21f9010 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:44.871118 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1368/2737] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:51:44.873010 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efecfd8-9220-4934-b102-d5bf7eb3b6b9 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:44.892953 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efecfd8-9220-4934-b102-d5bf7eb3b6b9 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:44.902159 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc530f-90aa-4845-b731-1ca2bc31a7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:44.903504 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc530f-90aa-4845-b731-1ca2bc31a7db admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:44.904113 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc530f-90aa-4845-b731-1ca2bc31a7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:44.904769 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc530f-90aa-4845-b731-1ca2bc31a7db admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:44.906148 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc530f-90aa-4845-b731-1ca2bc31a7db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:44.909993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc530f-90aa-4845-b731-1ca2bc31a7db admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:44.969417 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1370/2738] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:51:44 2022] POST /identity/v3/projects => generated 409 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:51:44.969979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3752c3-18b3-46a8-8ae5-2a7cb21f9010 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:44.983725 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3752c3-18b3-46a8-8ae5-2a7cb21f9010 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:45.060621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b6b0c-725b-4452-99e5-5200dd3b8df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:45.061794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b6b0c-725b-4452-99e5-5200dd3b8df2 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:45.062284 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b6b0c-725b-4452-99e5-5200dd3b8df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:45.062753 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b6b0c-725b-4452-99e5-5200dd3b8df2 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:45.063984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b6b0c-725b-4452-99e5-5200dd3b8df2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:45.068029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b6b0c-725b-4452-99e5-5200dd3b8df2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:45.104790 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1369/2739] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:51:44 2022] POST /identity/v3/users => generated 305 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:51:45.110432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc530f-90aa-4845-b731-1ca2bc31a7db admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:45.123818 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc530f-90aa-4845-b731-1ca2bc31a7db admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:45.129491 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4fda4-7f22-4cd7-b4b5-a89b6a362b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:45.130388 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4fda4-7f22-4cd7-b4b5-a89b6a362b52 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:45.130732 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4fda4-7f22-4cd7-b4b5-a89b6a362b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:45.131064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4fda4-7f22-4cd7-b4b5-a89b6a362b52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:45.131865 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4fda4-7f22-4cd7-b4b5-a89b6a362b52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:45.134172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4fda4-7f22-4cd7-b4b5-a89b6a362b52 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:45.145745 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1371/2740] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:51:45.150694 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b6b0c-725b-4452-99e5-5200dd3b8df2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:45.162568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b6b0c-725b-4452-99e5-5200dd3b8df2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:45.167631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26770c72-a5e8-4792-9501-de80029ae1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:45.168283 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26770c72-a5e8-4792-9501-de80029ae1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:45.168569 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26770c72-a5e8-4792-9501-de80029ae1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:45.168846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26770c72-a5e8-4792-9501-de80029ae1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:45.169616 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26770c72-a5e8-4792-9501-de80029ae1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:45.171273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26770c72-a5e8-4792-9501-de80029ae1b3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:45.187043 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1370/2741] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:51:45.191171 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4fda4-7f22-4cd7-b4b5-a89b6a362b52 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:45.203595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4fda4-7f22-4cd7-b4b5-a89b6a362b52 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:45.208921 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed81472-511c-4b35-b0b4-78291349f87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:45.209682 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed81472-511c-4b35-b0b4-78291349f87b admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:45.209958 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed81472-511c-4b35-b0b4-78291349f87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:45.210408 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed81472-511c-4b35-b0b4-78291349f87b admin admin] PATH_INFO: `/v3/projects/d89acde798254827989c0e549818e91a/users/725914134efa49d39cfe8fa529129825/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:45.235692 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed81472-511c-4b35-b0b4-78291349f87b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d89acde798254827989c0e549818e91a, user_id=725914134efa49d39cfe8fa529129825, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:45.238661 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed81472-511c-4b35-b0b4-78291349f87b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:45.255707 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1372/2742] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:51:45 2022] PUT /identity/v3/projects/d89acde798254827989c0e549818e91a/users/725914134efa49d39cfe8fa529129825/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:51:45.267402 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02243267-7510-4531-b43c-5f0a9b1ebf28 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:45.267786 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02243267-7510-4531-b43c-5f0a9b1ebf28 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:45.268870 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02243267-7510-4531-b43c-5f0a9b1ebf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:45.303031 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-02243267-7510-4531-b43c-5f0a9b1ebf28 None None] MFA Rules not processed for user `725914134efa49d39cfe8fa529129825`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:45.303374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-02243267-7510-4531-b43c-5f0a9b1ebf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:45.424715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1371/2743] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:45 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:51:45.426427 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed81472-511c-4b35-b0b4-78291349f87b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:45.432594 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:45.538607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed81472-511c-4b35-b0b4-78291349f87b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:45.563607 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d82cfd-b04f-4947-888e-67293208df9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:45.564326 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d82cfd-b04f-4947-888e-67293208df9f admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:45.564615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d82cfd-b04f-4947-888e-67293208df9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:45.564891 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d82cfd-b04f-4947-888e-67293208df9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:45.565642 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d82cfd-b04f-4947-888e-67293208df9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:45.567803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d82cfd-b04f-4947-888e-67293208df9f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:45.636466 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1373/2744] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:51:45 2022] POST /identity/v3/projects => generated 407 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:51:45.637716 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02243267-7510-4531-b43c-5f0a9b1ebf28 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:45.649171 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d82cfd-b04f-4947-888e-67293208df9f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:45.652164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-71d82cfd-b04f-4947-888e-67293208df9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:45.661699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02243267-7510-4531-b43c-5f0a9b1ebf28 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:45.703059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:45.759834 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b68ac-bfed-42b4-a552-fcd0cd3a5c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:45.760626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b68ac-bfed-42b4-a552-fcd0cd3a5c06 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:45.760981 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b68ac-bfed-42b4-a552-fcd0cd3a5c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:45.761326 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b68ac-bfed-42b4-a552-fcd0cd3a5c06 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:45.762127 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b68ac-bfed-42b4-a552-fcd0cd3a5c06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:45.764397 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b68ac-bfed-42b4-a552-fcd0cd3a5c06 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:45.803793 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1372/2745] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:51:45 2022] POST /identity/v3/users => generated 310 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:51:45.806977 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b68ac-bfed-42b4-a552-fcd0cd3a5c06 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:45.837245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b68ac-bfed-42b4-a552-fcd0cd3a5c06 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:45.844900 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967714b-553b-41b2-9a96-10651750f41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:45.846916 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967714b-553b-41b2-9a96-10651750f41e admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:45.847372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967714b-553b-41b2-9a96-10651750f41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:45.847850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967714b-553b-41b2-9a96-10651750f41e admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:45.848785 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967714b-553b-41b2-9a96-10651750f41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:45.850660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967714b-553b-41b2-9a96-10651750f41e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:45.852875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d82cfd-b04f-4947-888e-67293208df9f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:45.862719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72ec70-24e2-4165-99f9-d7019da1c6f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2c5536eddd346d692822a2f3befb48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:45.863874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72ec70-24e2-4165-99f9-d7019da1c6f7 service placement] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:45.863874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72ec70-24e2-4165-99f9-d7019da1c6f7 service placement] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:45.864193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72ec70-24e2-4165-99f9-d7019da1c6f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:45.868452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cd72ec70-24e2-4165-99f9-d7019da1c6f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:45.876493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1373/2746] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:51:45.878194 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967714b-553b-41b2-9a96-10651750f41e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:45.893369 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72ec70-24e2-4165-99f9-d7019da1c6f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:45.895054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72ec70-24e2-4165-99f9-d7019da1c6f7 service placement] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:45.897120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967714b-553b-41b2-9a96-10651750f41e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:45.906264 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d8aef-38cf-441f-b148-81a683fa0e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:45.907769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d8aef-38cf-441f-b148-81a683fa0e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:45.908407 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d8aef-38cf-441f-b148-81a683fa0e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:45.908874 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d8aef-38cf-441f-b148-81a683fa0e40 admin admin] PATH_INFO: `/v3/projects/3ebc6f3623604ae0948a8aea2ec5994a/users/089db109978b44c581c0fc65cf67aacf/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:45.910747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1374/2747] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:45.930962 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d8aef-38cf-441f-b148-81a683fa0e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ebc6f3623604ae0948a8aea2ec5994a, user_id=089db109978b44c581c0fc65cf67aacf, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:45.934535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d8aef-38cf-441f-b148-81a683fa0e40 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:45.951873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1374/2748] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:51:45 2022] PUT /identity/v3/projects/3ebc6f3623604ae0948a8aea2ec5994a/users/089db109978b44c581c0fc65cf67aacf/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:45.956189 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72ec70-24e2-4165-99f9-d7019da1c6f7 service placement] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:45.970253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72ec70-24e2-4165-99f9-d7019da1c6f7 service placement] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:46.033557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:46.042946 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dddd0bd-83c1-4c22-8cdb-de92b50d7e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:46.044064 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddd0bd-83c1-4c22-8cdb-de92b50d7e11 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:46.044885 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddd0bd-83c1-4c22-8cdb-de92b50d7e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:46.045717 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddd0bd-83c1-4c22-8cdb-de92b50d7e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:46.046949 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dddd0bd-83c1-4c22-8cdb-de92b50d7e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:46.049553 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dddd0bd-83c1-4c22-8cdb-de92b50d7e11 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:46.061987 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1375/2749] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:51:46.067445 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d8aef-38cf-441f-b148-81a683fa0e40 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:46.080982 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d8aef-38cf-441f-b148-81a683fa0e40 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:46.088001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd42f0-db2b-4f69-8c4a-f0f501820999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:46.089247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd42f0-db2b-4f69-8c4a-f0f501820999 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:46.089247 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd42f0-db2b-4f69-8c4a-f0f501820999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:46.089441 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd42f0-db2b-4f69-8c4a-f0f501820999 admin admin] PATH_INFO: `/v3/projects/3ebc6f3623604ae0948a8aea2ec5994a/users/089db109978b44c581c0fc65cf67aacf/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:46.091403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd42f0-db2b-4f69-8c4a-f0f501820999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ebc6f3623604ae0948a8aea2ec5994a, user_id=089db109978b44c581c0fc65cf67aacf, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:46.094368 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd42f0-db2b-4f69-8c4a-f0f501820999 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:46.106739 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1375/2750] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:51:46 2022] PUT /identity/v3/projects/3ebc6f3623604ae0948a8aea2ec5994a/users/089db109978b44c581c0fc65cf67aacf/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:46.120665 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1df6c-206e-402e-b79f-9f8ffe0a1410 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:46.121247 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1df6c-206e-402e-b79f-9f8ffe0a1410 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:46.121674 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1df6c-206e-402e-b79f-9f8ffe0a1410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:46.160844 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-cbf1df6c-206e-402e-b79f-9f8ffe0a1410 None None] MFA Rules not processed for user `089db109978b44c581c0fc65cf67aacf`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:46.162009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cbf1df6c-206e-402e-b79f-9f8ffe0a1410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:46.239532 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1376/2751] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:46 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:46.252330 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd42f0-db2b-4f69-8c4a-f0f501820999 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:46.254273 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9afd42f0-db2b-4f69-8c4a-f0f501820999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:46.318250 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd42f0-db2b-4f69-8c4a-f0f501820999 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:46.321995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0c073-3dd5-417d-897e-77bbaccb210f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:46.322631 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0c073-3dd5-417d-897e-77bbaccb210f service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:46.322912 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0c073-3dd5-417d-897e-77bbaccb210f service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:46.323218 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0c073-3dd5-417d-897e-77bbaccb210f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:46.333285 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a0c073-3dd5-417d-897e-77bbaccb210f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:46.334768 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a0c073-3dd5-417d-897e-77bbaccb210f service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:46.345633 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1376/2752] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:46 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:51:46.521507 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1df6c-206e-402e-b79f-9f8ffe0a1410 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:46.533537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1df6c-206e-402e-b79f-9f8ffe0a1410 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:46.537946 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750be846-64b8-4305-ad74-f0b3918f31bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:46.538807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750be846-64b8-4305-ad74-f0b3918f31bc service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:46.539211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750be846-64b8-4305-ad74-f0b3918f31bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:46.539615 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750be846-64b8-4305-ad74-f0b3918f31bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:46.606780 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750be846-64b8-4305-ad74-f0b3918f31bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:46.608264 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750be846-64b8-4305-ad74-f0b3918f31bc service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:46.619123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1377/2753] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:46 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:51:56.341034 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=61,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:56.614066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=65 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:59.531401 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0c073-3dd5-417d-897e-77bbaccb210f service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:59.533522 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-08a0c073-3dd5-417d-897e-77bbaccb210f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:59.615022 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750be846-64b8-4305-ad74-f0b3918f31bc service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:59.624446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0c073-3dd5-417d-897e-77bbaccb210f service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:59.630059 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfb906-681f-48b7-a0ec-adfa111780db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:59.630718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfb906-681f-48b7-a0ec-adfa111780db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:59.630998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfb906-681f-48b7-a0ec-adfa111780db admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:59.631206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750be846-64b8-4305-ad74-f0b3918f31bc service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:59.631364 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfb906-681f-48b7-a0ec-adfa111780db admin admin] PATH_INFO: `/v3/users/09c401e432344e0eb2204111e48c3730` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:59.632649 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dfb906-681f-48b7-a0ec-adfa111780db admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c401e432344e0eb2204111e48c3730)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:59.634826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dfb906-681f-48b7-a0ec-adfa111780db admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:59.637874 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc4842-acf0-484f-b375-231ea399003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:59.639283 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc4842-acf0-484f-b375-231ea399003a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:59.639771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc4842-acf0-484f-b375-231ea399003a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:59.640428 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc4842-acf0-484f-b375-231ea399003a admin admin] PATH_INFO: `/v3/users/725914134efa49d39cfe8fa529129825` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:59.642238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc4842-acf0-484f-b375-231ea399003a admin admin] RBAC: Authorizing `identity:delete_user(user_id=725914134efa49d39cfe8fa529129825)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:59.645139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc4842-acf0-484f-b375-231ea399003a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:59.717231 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d7dfb906-681f-48b7-a0ec-adfa111780db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c401e432344e0eb2204111e48c3730'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:59.719980 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-edcc4842-acf0-484f-b375-231ea399003a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '725914134efa49d39cfe8fa529129825'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:59.732540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-edcc4842-acf0-484f-b375-231ea399003a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '725914134efa49d39cfe8fa529129825'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:59.736044 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d7dfb906-681f-48b7-a0ec-adfa111780db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c401e432344e0eb2204111e48c3730'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:59.737055 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d7dfb906-681f-48b7-a0ec-adfa111780db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c401e432344e0eb2204111e48c3730'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:59.745086 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-edcc4842-acf0-484f-b375-231ea399003a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '725914134efa49d39cfe8fa529129825'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:59.746039 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-edcc4842-acf0-484f-b375-231ea399003a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '725914134efa49d39cfe8fa529129825'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:59.747138 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-edcc4842-acf0-484f-b375-231ea399003a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '725914134efa49d39cfe8fa529129825'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:59.750230 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d7dfb906-681f-48b7-a0ec-adfa111780db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c401e432344e0eb2204111e48c3730'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:59.761001 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d7dfb906-681f-48b7-a0ec-adfa111780db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c401e432344e0eb2204111e48c3730'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:51:59.771846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1377/2754] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:59 2022] DELETE /identity/v3/users/09c401e432344e0eb2204111e48c3730 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:51:59.776115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1378/2755] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:59 2022] DELETE /identity/v3/users/725914134efa49d39cfe8fa529129825 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:51:59.778681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0fcd3-26cb-40a9-b267-7599425c9d1d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:59.779149 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0fcd3-26cb-40a9-b267-7599425c9d1d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:59.779594 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0fcd3-26cb-40a9-b267-7599425c9d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:59.781340 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60288e79-4122-4c4c-9c31-1e8146c82ff7 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:59.781842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60288e79-4122-4c4c-9c31-1e8146c82ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:59.782300 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60288e79-4122-4c4c-9c31-1e8146c82ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:59.809755 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-60288e79-4122-4c4c-9c31-1e8146c82ff7 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:59.811044 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-60288e79-4122-4c4c-9c31-1e8146c82ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:59.811776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-aaa0fcd3-26cb-40a9-b267-7599425c9d1d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:51:59.813311 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-aaa0fcd3-26cb-40a9-b267-7599425c9d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:59.893546 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1378/2756] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:59.894219 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0fcd3-26cb-40a9-b267-7599425c9d1d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:59.895784 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-aaa0fcd3-26cb-40a9-b267-7599425c9d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:59.899313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1379/2757] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:51:59.908118 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60288e79-4122-4c4c-9c31-1e8146c82ff7 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:51:59.910926 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-60288e79-4122-4c4c-9c31-1e8146c82ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:51:59.913548 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0fcd3-26cb-40a9-b267-7599425c9d1d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:51:59.918797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf2a18-c635-4170-81c0-822b0f7fbfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:51:59.919560 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf2a18-c635-4170-81c0-822b0f7fbfd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:51:59.919873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf2a18-c635-4170-81c0-822b0f7fbfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:51:59.920140 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf2a18-c635-4170-81c0-822b0f7fbfd1 admin admin] PATH_INFO: `/v3/users/7aa23dc4d6614cedbd22db551682860e` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:51:59.921435 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf2a18-c635-4170-81c0-822b0f7fbfd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aa23dc4d6614cedbd22db551682860e)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:51:59.923464 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf2a18-c635-4170-81c0-822b0f7fbfd1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:51:59.943356 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:51:59.994309 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-92bf2a18-c635-4170-81c0-822b0f7fbfd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aa23dc4d6614cedbd22db551682860e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:00.011109 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-92bf2a18-c635-4170-81c0-822b0f7fbfd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aa23dc4d6614cedbd22db551682860e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:00.012068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-92bf2a18-c635-4170-81c0-822b0f7fbfd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aa23dc4d6614cedbd22db551682860e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:00.023589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-92bf2a18-c635-4170-81c0-822b0f7fbfd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aa23dc4d6614cedbd22db551682860e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:00.033640 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-92bf2a18-c635-4170-81c0-822b0f7fbfd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aa23dc4d6614cedbd22db551682860e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:00.038659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60288e79-4122-4c4c-9c31-1e8146c82ff7 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:00.042430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1379/2758] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:51:59 2022] DELETE /identity/v3/users/7aa23dc4d6614cedbd22db551682860e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:00.043611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf2a18-c635-4170-81c0-822b0f7fbfd1 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:00.044807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825d26e-0e89-433d-8eaf-7d345ddb7b74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:00.045861 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-92bf2a18-c635-4170-81c0-822b0f7fbfd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:00.055795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825d26e-0e89-433d-8eaf-7d345ddb7b74 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:00.056287 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825d26e-0e89-433d-8eaf-7d345ddb7b74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:00.056574 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825d26e-0e89-433d-8eaf-7d345ddb7b74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:00.060039 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6825d26e-0e89-433d-8eaf-7d345ddb7b74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:00.074501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:00.077043 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825d26e-0e89-433d-8eaf-7d345ddb7b74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:00.078562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825d26e-0e89-433d-8eaf-7d345ddb7b74 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:00.093905 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1380/2759] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:52:00.095526 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37f472-209f-4b7a-b07d-ea27411d52b7 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:00.095817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37f472-209f-4b7a-b07d-ea27411d52b7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:00.096265 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37f472-209f-4b7a-b07d-ea27411d52b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:00.118356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-1d37f472-209f-4b7a-b07d-ea27411d52b7 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:00.119778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1d37f472-209f-4b7a-b07d-ea27411d52b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:00.163883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf2a18-c635-4170-81c0-822b0f7fbfd1 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:00.172072 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3bf29-b728-4eb0-bbac-adf4c1b22a2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:00.173835 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3bf29-b728-4eb0-bbac-adf4c1b22a2e service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:00.174601 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3bf29-b728-4eb0-bbac-adf4c1b22a2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:00.174910 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3bf29-b728-4eb0-bbac-adf4c1b22a2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:00.177902 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-41a3bf29-b728-4eb0-bbac-adf4c1b22a2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:00.220873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1381/2760] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:00.233631 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37f472-209f-4b7a-b07d-ea27411d52b7 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:00.253411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37f472-209f-4b7a-b07d-ea27411d52b7 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:00.262104 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722b8b3-acd7-4a75-b5c8-2c4944bb7faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:00.262907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722b8b3-acd7-4a75-b5c8-2c4944bb7faf service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:00.263104 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722b8b3-acd7-4a75-b5c8-2c4944bb7faf service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:00.263290 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722b8b3-acd7-4a75-b5c8-2c4944bb7faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:00.274318 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722b8b3-acd7-4a75-b5c8-2c4944bb7faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:00.275957 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722b8b3-acd7-4a75-b5c8-2c4944bb7faf service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:00.281269 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3bf29-b728-4eb0-bbac-adf4c1b22a2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:00.282875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3bf29-b728-4eb0-bbac-adf4c1b22a2e service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:00.287697 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1382/2761] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:52:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:52:00.302415 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1380/2762] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:52:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:52:00.479073 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722b8b3-acd7-4a75-b5c8-2c4944bb7faf service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:00.480652 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0722b8b3-acd7-4a75-b5c8-2c4944bb7faf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:00.511415 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722b8b3-acd7-4a75-b5c8-2c4944bb7faf service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:00.522094 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d215f-2934-4b63-9561-0320014f2219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:00.523843 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d215f-2934-4b63-9561-0320014f2219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:00.524250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d215f-2934-4b63-9561-0320014f2219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:00.524533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d215f-2934-4b63-9561-0320014f2219 admin admin] PATH_INFO: `/v3/projects/a1a3251710054d589ceeee91cf556bbc` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:00.526340 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d215f-2934-4b63-9561-0320014f2219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a3251710054d589ceeee91cf556bbc)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:00.528323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d215f-2934-4b63-9561-0320014f2219 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:00.602657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-001d215f-2934-4b63-9561-0320014f2219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a3251710054d589ceeee91cf556bbc', 'request_id': 'req-001d215f-2934-4b63-9561-0320014f2219'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:00.611545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-001d215f-2934-4b63-9561-0320014f2219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a3251710054d589ceeee91cf556bbc', 'request_id': 'req-001d215f-2934-4b63-9561-0320014f2219'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:00.627855 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-001d215f-2934-4b63-9561-0320014f2219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a3251710054d589ceeee91cf556bbc', 'request_id': 'req-001d215f-2934-4b63-9561-0320014f2219'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:00.652085 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-001d215f-2934-4b63-9561-0320014f2219 admin admin] The token cache is being invalidate because project a1a3251710054d589ceeee91cf556bbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:52:00.652614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-001d215f-2934-4b63-9561-0320014f2219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:00.653698 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-001d215f-2934-4b63-9561-0320014f2219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:00.655803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1383/2763] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:52:00 2022] DELETE /identity/v3/projects/a1a3251710054d589ceeee91cf556bbc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:00.859418 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3bf29-b728-4eb0-bbac-adf4c1b22a2e service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:00.862480 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-41a3bf29-b728-4eb0-bbac-adf4c1b22a2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:00.893656 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d215f-2934-4b63-9561-0320014f2219 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:00.896135 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-001d215f-2934-4b63-9561-0320014f2219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:00.921800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3bf29-b728-4eb0-bbac-adf4c1b22a2e service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:00.996803 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d215f-2934-4b63-9561-0320014f2219 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:01.005192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f1aab-a51e-413c-be9e-d18c4e829ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:01.006229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f1aab-a51e-413c-be9e-d18c4e829ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:01.006407 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:01.006663 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f1aab-a51e-413c-be9e-d18c4e829ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:01.007048 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f1aab-a51e-413c-be9e-d18c4e829ecb admin admin] PATH_INFO: `/v3/projects/d89acde798254827989c0e549818e91a` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:01.009393 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f1aab-a51e-413c-be9e-d18c4e829ecb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d89acde798254827989c0e549818e91a)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:01.011826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f1aab-a51e-413c-be9e-d18c4e829ecb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:01.022016 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:01.033692 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019388c-99af-4c73-a2f2-00191b467af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:01.034842 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019388c-99af-4c73-a2f2-00191b467af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:01.035132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019388c-99af-4c73-a2f2-00191b467af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:01.035409 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019388c-99af-4c73-a2f2-00191b467af3 admin admin] PATH_INFO: `/v3/projects/46f6e58889084b2ab7cb010c9c9c704f` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:01.037033 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019388c-99af-4c73-a2f2-00191b467af3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f6e58889084b2ab7cb010c9c9c704f)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:01.039617 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019388c-99af-4c73-a2f2-00191b467af3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:01.075562 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-403f1aab-a51e-413c-be9e-d18c4e829ecb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd89acde798254827989c0e549818e91a', 'request_id': 'req-403f1aab-a51e-413c-be9e-d18c4e829ecb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:01.088605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-403f1aab-a51e-413c-be9e-d18c4e829ecb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd89acde798254827989c0e549818e91a', 'request_id': 'req-403f1aab-a51e-413c-be9e-d18c4e829ecb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:01.100054 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-403f1aab-a51e-413c-be9e-d18c4e829ecb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd89acde798254827989c0e549818e91a', 'request_id': 'req-403f1aab-a51e-413c-be9e-d18c4e829ecb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:01.102080 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4019388c-99af-4c73-a2f2-00191b467af3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f6e58889084b2ab7cb010c9c9c704f', 'request_id': 'req-4019388c-99af-4c73-a2f2-00191b467af3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:01.113982 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4019388c-99af-4c73-a2f2-00191b467af3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f6e58889084b2ab7cb010c9c9c704f', 'request_id': 'req-4019388c-99af-4c73-a2f2-00191b467af3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:01.118356 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-403f1aab-a51e-413c-be9e-d18c4e829ecb admin admin] The token cache is being invalidate because project d89acde798254827989c0e549818e91a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:52:01.118795 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-403f1aab-a51e-413c-be9e-d18c4e829ecb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:01.119678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-403f1aab-a51e-413c-be9e-d18c4e829ecb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:01.121363 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1384/2764] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:52:00 2022] DELETE /identity/v3/projects/d89acde798254827989c0e549818e91a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:01.125571 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f1aab-a51e-413c-be9e-d18c4e829ecb admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:01.127342 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4019388c-99af-4c73-a2f2-00191b467af3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f6e58889084b2ab7cb010c9c9c704f', 'request_id': 'req-4019388c-99af-4c73-a2f2-00191b467af3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:01.127608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-403f1aab-a51e-413c-be9e-d18c4e829ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:01.141145 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4019388c-99af-4c73-a2f2-00191b467af3 admin admin] The token cache is being invalidate because project 46f6e58889084b2ab7cb010c9c9c704f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:52:01.141572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4019388c-99af-4c73-a2f2-00191b467af3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:01.142266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4019388c-99af-4c73-a2f2-00191b467af3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:01.143882 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1381/2765] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:52:00 2022] DELETE /identity/v3/projects/46f6e58889084b2ab7cb010c9c9c704f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:01.248128 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f1aab-a51e-413c-be9e-d18c4e829ecb admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:01.343018 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f08a96-f47d-4e77-8d22-c460342bd769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:01.343721 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f08a96-f47d-4e77-8d22-c460342bd769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:01.344006 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f08a96-f47d-4e77-8d22-c460342bd769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:01.344293 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f08a96-f47d-4e77-8d22-c460342bd769 admin admin] PATH_INFO: `/v3/users/089db109978b44c581c0fc65cf67aacf` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:01.345708 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f08a96-f47d-4e77-8d22-c460342bd769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=089db109978b44c581c0fc65cf67aacf)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:01.347770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f08a96-f47d-4e77-8d22-c460342bd769 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:01.431327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:01.433763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-25f08a96-f47d-4e77-8d22-c460342bd769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '089db109978b44c581c0fc65cf67aacf'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:01.450169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-25f08a96-f47d-4e77-8d22-c460342bd769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '089db109978b44c581c0fc65cf67aacf'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:01.467147 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-25f08a96-f47d-4e77-8d22-c460342bd769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '089db109978b44c581c0fc65cf67aacf'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:01.468134 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-25f08a96-f47d-4e77-8d22-c460342bd769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '089db109978b44c581c0fc65cf67aacf'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:01.470723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-25f08a96-f47d-4e77-8d22-c460342bd769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '089db109978b44c581c0fc65cf67aacf'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:01.498711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1385/2766] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:52:01 2022] DELETE /identity/v3/users/089db109978b44c581c0fc65cf67aacf => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:52:01.860929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019388c-99af-4c73-a2f2-00191b467af3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:01.863187 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4019388c-99af-4c73-a2f2-00191b467af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:01.886723 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019388c-99af-4c73-a2f2-00191b467af3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:01.987526 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960d0d4-edfb-4092-bdc6-303775ab06de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:01.988198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960d0d4-edfb-4092-bdc6-303775ab06de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:01.988484 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960d0d4-edfb-4092-bdc6-303775ab06de admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:01.988766 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960d0d4-edfb-4092-bdc6-303775ab06de admin admin] PATH_INFO: `/v3/projects/3ebc6f3623604ae0948a8aea2ec5994a` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:01.990084 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960d0d4-edfb-4092-bdc6-303775ab06de admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ebc6f3623604ae0948a8aea2ec5994a)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:01.992359 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960d0d4-edfb-4092-bdc6-303775ab06de admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:02.050547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e960d0d4-edfb-4092-bdc6-303775ab06de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ebc6f3623604ae0948a8aea2ec5994a', 'request_id': 'req-e960d0d4-edfb-4092-bdc6-303775ab06de'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:02.060271 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e960d0d4-edfb-4092-bdc6-303775ab06de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ebc6f3623604ae0948a8aea2ec5994a', 'request_id': 'req-e960d0d4-edfb-4092-bdc6-303775ab06de'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:02.068260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e960d0d4-edfb-4092-bdc6-303775ab06de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ebc6f3623604ae0948a8aea2ec5994a', 'request_id': 'req-e960d0d4-edfb-4092-bdc6-303775ab06de'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:02.079978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e960d0d4-edfb-4092-bdc6-303775ab06de admin admin] The token cache is being invalidate because project 3ebc6f3623604ae0948a8aea2ec5994a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:52:02.080274 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e960d0d4-edfb-4092-bdc6-303775ab06de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:02.080940 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e960d0d4-edfb-4092-bdc6-303775ab06de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:02.082512 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1382/2767] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:52:01 2022] DELETE /identity/v3/projects/3ebc6f3623604ae0948a8aea2ec5994a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:02.087975 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e1b38-79d8-4d68-a7d3-f71cd0b23f3f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:02.088528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e1b38-79d8-4d68-a7d3-f71cd0b23f3f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:02.088528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e1b38-79d8-4d68-a7d3-f71cd0b23f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:02.111884 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7a0e1b38-79d8-4d68-a7d3-f71cd0b23f3f None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:02.113165 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7a0e1b38-79d8-4d68-a7d3-f71cd0b23f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:02.180700 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1386/2768] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:02.194278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0035fca-d2ce-49b5-bf87-7bcbb0b06d4d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:02.194776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0035fca-d2ce-49b5-bf87-7bcbb0b06d4d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:02.195780 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0035fca-d2ce-49b5-bf87-7bcbb0b06d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:02.208052 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:02.224307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f0035fca-d2ce-49b5-bf87-7bcbb0b06d4d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:02.225796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f0035fca-d2ce-49b5-bf87-7bcbb0b06d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:02.234771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1383/2769] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:52:02.240335 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e1b38-79d8-4d68-a7d3-f71cd0b23f3f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:02.254807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e1b38-79d8-4d68-a7d3-f71cd0b23f3f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:02.260141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfc12f-ed73-41e4-a13c-cbaef1fd1ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:02.260778 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfc12f-ed73-41e4-a13c-cbaef1fd1ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:02.261057 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfc12f-ed73-41e4-a13c-cbaef1fd1ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:02.261324 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfc12f-ed73-41e4-a13c-cbaef1fd1ad5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:02.262069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfc12f-ed73-41e4-a13c-cbaef1fd1ad5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:52:02.262368 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfc12f-ed73-41e4-a13c-cbaef1fd1ad5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:02.263998 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfc12f-ed73-41e4-a13c-cbaef1fd1ad5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:02.277434 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1387/2770] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 10:52:02.282053 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0035fca-d2ce-49b5-bf87-7bcbb0b06d4d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:02.291746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0035fca-d2ce-49b5-bf87-7bcbb0b06d4d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:02.296606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73437c-ef6f-4c41-ada5-212cfc7f496e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:02.297239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73437c-ef6f-4c41-ada5-212cfc7f496e admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:02.297592 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73437c-ef6f-4c41-ada5-212cfc7f496e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:02.297830 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73437c-ef6f-4c41-ada5-212cfc7f496e admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:02.298548 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73437c-ef6f-4c41-ada5-212cfc7f496e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:02.300421 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73437c-ef6f-4c41-ada5-212cfc7f496e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:02.337618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1384/2771] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:52:02 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:52:02.341817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfc12f-ed73-41e4-a13c-cbaef1fd1ad5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:02.353429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfc12f-ed73-41e4-a13c-cbaef1fd1ad5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:02.418058 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f83d30-73a8-4858-80ef-a1c9bda67665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:02.418694 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f83d30-73a8-4858-80ef-a1c9bda67665 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:02.418971 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f83d30-73a8-4858-80ef-a1c9bda67665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:02.419250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f83d30-73a8-4858-80ef-a1c9bda67665 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:02.420271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f83d30-73a8-4858-80ef-a1c9bda67665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:02.422296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f83d30-73a8-4858-80ef-a1c9bda67665 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:02.444675 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1388/2772] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:52:02 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:52:02.449348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73437c-ef6f-4c41-ada5-212cfc7f496e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:02.461616 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73437c-ef6f-4c41-ada5-212cfc7f496e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:02.467159 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede6242-da88-40a2-899e-e483d021ba4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:02.467794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede6242-da88-40a2-899e-e483d021ba4e admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:02.468072 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede6242-da88-40a2-899e-e483d021ba4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:02.468346 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede6242-da88-40a2-899e-e483d021ba4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:02.469050 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede6242-da88-40a2-899e-e483d021ba4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:02.470662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede6242-da88-40a2-899e-e483d021ba4e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:02.482794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1385/2773] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:52:02.487083 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f83d30-73a8-4858-80ef-a1c9bda67665 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:02.498486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f83d30-73a8-4858-80ef-a1c9bda67665 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:02.503248 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c1777-f372-4027-bfeb-fabbf6121169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:02.503872 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c1777-f372-4027-bfeb-fabbf6121169 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:02.504152 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c1777-f372-4027-bfeb-fabbf6121169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:02.504424 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c1777-f372-4027-bfeb-fabbf6121169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:02.505385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c1777-f372-4027-bfeb-fabbf6121169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:02.507038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c1777-f372-4027-bfeb-fabbf6121169 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:02.518654 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1389/2774] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:52:02.522954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede6242-da88-40a2-899e-e483d021ba4e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:02.534083 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede6242-da88-40a2-899e-e483d021ba4e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:02.539902 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2494440-ee7c-47e5-8731-d7b5d9b2d0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:02.540682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2494440-ee7c-47e5-8731-d7b5d9b2d0c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:02.541096 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2494440-ee7c-47e5-8731-d7b5d9b2d0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:02.541422 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2494440-ee7c-47e5-8731-d7b5d9b2d0c7 admin admin] PATH_INFO: `/v3/projects/0b63db1457364a4180ca8bae0147f0fe/users/a1dccbd1efa74121871f80bd62d19d8f/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:02.558101 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2494440-ee7c-47e5-8731-d7b5d9b2d0c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b63db1457364a4180ca8bae0147f0fe, user_id=a1dccbd1efa74121871f80bd62d19d8f, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:02.560848 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2494440-ee7c-47e5-8731-d7b5d9b2d0c7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:02.575905 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1386/2775] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:52:02 2022] PUT /identity/v3/projects/0b63db1457364a4180ca8bae0147f0fe/users/a1dccbd1efa74121871f80bd62d19d8f/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:02.587338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d040113-ef64-4541-b20d-c7f94849fdc3 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:02.587705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d040113-ef64-4541-b20d-c7f94849fdc3 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:02.587920 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d040113-ef64-4541-b20d-c7f94849fdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:02.615362 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2d040113-ef64-4541-b20d-c7f94849fdc3 None None] MFA Rules not processed for user `a1dccbd1efa74121871f80bd62d19d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:02.617371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2d040113-ef64-4541-b20d-c7f94849fdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:02.635789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:02.701139 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1390/2776] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:02.705738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2494440-ee7c-47e5-8731-d7b5d9b2d0c7 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:02.774472 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2494440-ee7c-47e5-8731-d7b5d9b2d0c7 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:02.779398 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285152b6-0786-4704-9c29-347d9bd35d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:02.781681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285152b6-0786-4704-9c29-347d9bd35d71 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:02.781681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285152b6-0786-4704-9c29-347d9bd35d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:02.781681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285152b6-0786-4704-9c29-347d9bd35d71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:02.781681 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285152b6-0786-4704-9c29-347d9bd35d71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:02.783179 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285152b6-0786-4704-9c29-347d9bd35d71 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:02.812388 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1387/2777] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:52:02 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:52:02.816523 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d040113-ef64-4541-b20d-c7f94849fdc3 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:02.828237 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d040113-ef64-4541-b20d-c7f94849fdc3 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:02.891077 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b85bb8-22ce-437b-9de3-43a0c84bdff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:02.891621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b85bb8-22ce-437b-9de3-43a0c84bdff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:02.891923 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b85bb8-22ce-437b-9de3-43a0c84bdff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:02.892193 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b85bb8-22ce-437b-9de3-43a0c84bdff5 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:02.892867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b85bb8-22ce-437b-9de3-43a0c84bdff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:02.894867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b85bb8-22ce-437b-9de3-43a0c84bdff5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:02.917794 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1391/2778] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:52:02 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:52:02.922822 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285152b6-0786-4704-9c29-347d9bd35d71 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:02.934710 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285152b6-0786-4704-9c29-347d9bd35d71 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:02.939487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76096c28-5215-44d8-b66b-37c13a654f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:02.940113 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76096c28-5215-44d8-b66b-37c13a654f76 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:02.940392 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76096c28-5215-44d8-b66b-37c13a654f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:02.940666 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76096c28-5215-44d8-b66b-37c13a654f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:02.941364 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76096c28-5215-44d8-b66b-37c13a654f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:02.942984 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76096c28-5215-44d8-b66b-37c13a654f76 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:03.088109 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1388/2779] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:52:03.089432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b85bb8-22ce-437b-9de3-43a0c84bdff5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:03.100095 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b85bb8-22ce-437b-9de3-43a0c84bdff5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:03.106555 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cce483-4f99-4e67-930c-ee8db4575edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:03.107336 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cce483-4f99-4e67-930c-ee8db4575edc admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:03.107782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cce483-4f99-4e67-930c-ee8db4575edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:03.108098 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cce483-4f99-4e67-930c-ee8db4575edc admin admin] PATH_INFO: `/v3/projects/c2aea0eec9464f21b31cdcaa76d47f3b/users/081b71df40bc434aa14a33818e70fc55/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:03.124927 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cce483-4f99-4e67-930c-ee8db4575edc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2aea0eec9464f21b31cdcaa76d47f3b, user_id=081b71df40bc434aa14a33818e70fc55, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:03.127656 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cce483-4f99-4e67-930c-ee8db4575edc admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:03.142837 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1392/2780] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:52:03 2022] PUT /identity/v3/projects/c2aea0eec9464f21b31cdcaa76d47f3b/users/081b71df40bc434aa14a33818e70fc55/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:52:03.146839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76096c28-5215-44d8-b66b-37c13a654f76 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:03.157329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76096c28-5215-44d8-b66b-37c13a654f76 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:03.240724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac065bc-e05d-4adc-b586-ead2e66c5ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:03.242622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac065bc-e05d-4adc-b586-ead2e66c5ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:03.243134 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac065bc-e05d-4adc-b586-ead2e66c5ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:03.243645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac065bc-e05d-4adc-b586-ead2e66c5ab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:03.245076 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac065bc-e05d-4adc-b586-ead2e66c5ab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:03.248237 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac065bc-e05d-4adc-b586-ead2e66c5ab8 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:03.267732 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1389/2781] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:52:03.272393 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cce483-4f99-4e67-930c-ee8db4575edc admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:03.284024 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cce483-4f99-4e67-930c-ee8db4575edc admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:03.289254 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244843e0-f7b6-47fd-9fe9-958f1c39f660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:03.290055 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244843e0-f7b6-47fd-9fe9-958f1c39f660 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:03.290344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244843e0-f7b6-47fd-9fe9-958f1c39f660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:03.290629 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244843e0-f7b6-47fd-9fe9-958f1c39f660 admin admin] PATH_INFO: `/v3/projects/c2aea0eec9464f21b31cdcaa76d47f3b/users/081b71df40bc434aa14a33818e70fc55/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:03.292689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244843e0-f7b6-47fd-9fe9-958f1c39f660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2aea0eec9464f21b31cdcaa76d47f3b, user_id=081b71df40bc434aa14a33818e70fc55, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:03.295576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244843e0-f7b6-47fd-9fe9-958f1c39f660 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:03.308251 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1393/2782] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:52:03 2022] PUT /identity/v3/projects/c2aea0eec9464f21b31cdcaa76d47f3b/users/081b71df40bc434aa14a33818e70fc55/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:03.319000 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8954f-fab1-48c2-9ce8-187b4ecb4a44 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:03.319318 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8954f-fab1-48c2-9ce8-187b4ecb4a44 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:03.319572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8954f-fab1-48c2-9ce8-187b4ecb4a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:03.340576 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:03.352408 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-92e8954f-fab1-48c2-9ce8-187b4ecb4a44 None None] MFA Rules not processed for user `081b71df40bc434aa14a33818e70fc55`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:03.354533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-92e8954f-fab1-48c2-9ce8-187b4ecb4a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:03.427655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1390/2783] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:03.439929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244843e0-f7b6-47fd-9fe9-958f1c39f660 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:03.441644 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-244843e0-f7b6-47fd-9fe9-958f1c39f660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:03.520186 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244843e0-f7b6-47fd-9fe9-958f1c39f660 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:03.523886 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44b2ac-a62d-4e8a-97e9-1dba14046608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:03.524501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44b2ac-a62d-4e8a-97e9-1dba14046608 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:03.524783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44b2ac-a62d-4e8a-97e9-1dba14046608 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:03.525057 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44b2ac-a62d-4e8a-97e9-1dba14046608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:03.530973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:03.534423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44b2ac-a62d-4e8a-97e9-1dba14046608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:03.536160 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44b2ac-a62d-4e8a-97e9-1dba14046608 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:03.546589 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1394/2784] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:52:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:52:03.618646 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8954f-fab1-48c2-9ce8-187b4ecb4a44 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:03.628434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8954f-fab1-48c2-9ce8-187b4ecb4a44 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:03.632209 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32039fde-d046-4738-8f9f-c934d50006b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:03.633408 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32039fde-d046-4738-8f9f-c934d50006b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:03.633721 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32039fde-d046-4738-8f9f-c934d50006b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:03.634005 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32039fde-d046-4738-8f9f-c934d50006b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:03.691745 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32039fde-d046-4738-8f9f-c934d50006b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:03.693163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32039fde-d046-4738-8f9f-c934d50006b3 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:03.702148 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1391/2785] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:52:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:52:03.963776 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44b2ac-a62d-4e8a-97e9-1dba14046608 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:04.025659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44b2ac-a62d-4e8a-97e9-1dba14046608 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:04.030034 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece7430-730b-4111-9cc4-003723423abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:04.030767 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece7430-730b-4111-9cc4-003723423abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:04.031049 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece7430-730b-4111-9cc4-003723423abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:04.031321 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece7430-730b-4111-9cc4-003723423abd admin admin] PATH_INFO: `/v3/users/a1dccbd1efa74121871f80bd62d19d8f` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:04.032463 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece7430-730b-4111-9cc4-003723423abd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1dccbd1efa74121871f80bd62d19d8f)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:04.034406 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece7430-730b-4111-9cc4-003723423abd admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:04.080334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-cece7430-730b-4111-9cc4-003723423abd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1dccbd1efa74121871f80bd62d19d8f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:04.088189 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-cece7430-730b-4111-9cc4-003723423abd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1dccbd1efa74121871f80bd62d19d8f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:04.096192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-cece7430-730b-4111-9cc4-003723423abd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1dccbd1efa74121871f80bd62d19d8f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:04.096794 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-cece7430-730b-4111-9cc4-003723423abd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1dccbd1efa74121871f80bd62d19d8f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:04.097408 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-cece7430-730b-4111-9cc4-003723423abd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1dccbd1efa74121871f80bd62d19d8f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:04.115374 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1395/2786] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:52:03 2022] DELETE /identity/v3/users/a1dccbd1efa74121871f80bd62d19d8f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:52:04.120191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727063c5-c884-444b-bc1d-003fcebab318 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:04.120648 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727063c5-c884-444b-bc1d-003fcebab318 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:04.120748 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727063c5-c884-444b-bc1d-003fcebab318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:04.139212 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-727063c5-c884-444b-bc1d-003fcebab318 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:04.140405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-727063c5-c884-444b-bc1d-003fcebab318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:04.206460 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1392/2787] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:04.218684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece7430-730b-4111-9cc4-003723423abd admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:04.220100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cece7430-730b-4111-9cc4-003723423abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:04.290619 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece7430-730b-4111-9cc4-003723423abd admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:04.296595 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361f1d6-a167-427e-ab66-157a5540d679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:04.297682 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361f1d6-a167-427e-ab66-157a5540d679 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:04.298183 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361f1d6-a167-427e-ab66-157a5540d679 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:04.298659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361f1d6-a167-427e-ab66-157a5540d679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:04.313987 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361f1d6-a167-427e-ab66-157a5540d679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:04.315421 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361f1d6-a167-427e-ab66-157a5540d679 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:04.325653 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1396/2788] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:52:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:52:04.634188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727063c5-c884-444b-bc1d-003fcebab318 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:04.635656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-727063c5-c884-444b-bc1d-003fcebab318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:04.652314 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727063c5-c884-444b-bc1d-003fcebab318 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:04.657399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508934b-30d0-47ab-8aa2-b8444d7e13b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:04.658198 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508934b-30d0-47ab-8aa2-b8444d7e13b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:04.658554 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508934b-30d0-47ab-8aa2-b8444d7e13b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:04.658850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508934b-30d0-47ab-8aa2-b8444d7e13b1 admin admin] PATH_INFO: `/v3/projects/0b63db1457364a4180ca8bae0147f0fe` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:04.660077 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508934b-30d0-47ab-8aa2-b8444d7e13b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b63db1457364a4180ca8bae0147f0fe)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:04.661966 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508934b-30d0-47ab-8aa2-b8444d7e13b1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:04.698591 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:04.724979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3508934b-30d0-47ab-8aa2-b8444d7e13b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b63db1457364a4180ca8bae0147f0fe', 'request_id': 'req-3508934b-30d0-47ab-8aa2-b8444d7e13b1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:04.735683 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3508934b-30d0-47ab-8aa2-b8444d7e13b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b63db1457364a4180ca8bae0147f0fe', 'request_id': 'req-3508934b-30d0-47ab-8aa2-b8444d7e13b1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:04.743926 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3508934b-30d0-47ab-8aa2-b8444d7e13b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b63db1457364a4180ca8bae0147f0fe', 'request_id': 'req-3508934b-30d0-47ab-8aa2-b8444d7e13b1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:04.754928 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3508934b-30d0-47ab-8aa2-b8444d7e13b1 admin admin] The token cache is being invalidate because project 0b63db1457364a4180ca8bae0147f0fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:52:04.755210 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3508934b-30d0-47ab-8aa2-b8444d7e13b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:04.755878 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3508934b-30d0-47ab-8aa2-b8444d7e13b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:04.758063 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1393/2789] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:52:04 2022] DELETE /identity/v3/projects/0b63db1457364a4180ca8bae0147f0fe => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:04.762482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361f1d6-a167-427e-ab66-157a5540d679 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:04.772552 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361f1d6-a167-427e-ab66-157a5540d679 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:04.785271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:04.835440 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2c924-5b2c-42fb-b49d-b1bd489f7b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:04.836137 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2c924-5b2c-42fb-b49d-b1bd489f7b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:04.836442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2c924-5b2c-42fb-b49d-b1bd489f7b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:04.836719 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2c924-5b2c-42fb-b49d-b1bd489f7b07 admin admin] PATH_INFO: `/v3/users/081b71df40bc434aa14a33818e70fc55` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:04.837935 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2c924-5b2c-42fb-b49d-b1bd489f7b07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=081b71df40bc434aa14a33818e70fc55)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:04.840147 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2c924-5b2c-42fb-b49d-b1bd489f7b07 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:04.886589 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5d2c924-5b2c-42fb-b49d-b1bd489f7b07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '081b71df40bc434aa14a33818e70fc55'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:04.893838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5d2c924-5b2c-42fb-b49d-b1bd489f7b07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '081b71df40bc434aa14a33818e70fc55'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:04.901310 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5d2c924-5b2c-42fb-b49d-b1bd489f7b07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '081b71df40bc434aa14a33818e70fc55'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:04.901957 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5d2c924-5b2c-42fb-b49d-b1bd489f7b07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '081b71df40bc434aa14a33818e70fc55'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:04.902527 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5d2c924-5b2c-42fb-b49d-b1bd489f7b07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '081b71df40bc434aa14a33818e70fc55'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:04.919760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1397/2790] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:52:04 2022] DELETE /identity/v3/users/081b71df40bc434aa14a33818e70fc55 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:05.190265 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508934b-30d0-47ab-8aa2-b8444d7e13b1 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:05.191749 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3508934b-30d0-47ab-8aa2-b8444d7e13b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:05.204669 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508934b-30d0-47ab-8aa2-b8444d7e13b1 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:05.264178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e0bc3-ad43-446c-a6fb-9a697d5fa27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:05.264755 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e0bc3-ad43-446c-a6fb-9a697d5fa27f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:05.265040 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e0bc3-ad43-446c-a6fb-9a697d5fa27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:05.265315 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e0bc3-ad43-446c-a6fb-9a697d5fa27f admin admin] PATH_INFO: `/v3/projects/c2aea0eec9464f21b31cdcaa76d47f3b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:05.266509 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e0bc3-ad43-446c-a6fb-9a697d5fa27f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2aea0eec9464f21b31cdcaa76d47f3b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:05.268384 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e0bc3-ad43-446c-a6fb-9a697d5fa27f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:05.315058 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8c6e0bc3-ad43-446c-a6fb-9a697d5fa27f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2aea0eec9464f21b31cdcaa76d47f3b', 'request_id': 'req-8c6e0bc3-ad43-446c-a6fb-9a697d5fa27f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:05.322238 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8c6e0bc3-ad43-446c-a6fb-9a697d5fa27f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2aea0eec9464f21b31cdcaa76d47f3b', 'request_id': 'req-8c6e0bc3-ad43-446c-a6fb-9a697d5fa27f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:05.328393 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8c6e0bc3-ad43-446c-a6fb-9a697d5fa27f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2aea0eec9464f21b31cdcaa76d47f3b', 'request_id': 'req-8c6e0bc3-ad43-446c-a6fb-9a697d5fa27f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:05.338711 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8c6e0bc3-ad43-446c-a6fb-9a697d5fa27f admin admin] The token cache is being invalidate because project c2aea0eec9464f21b31cdcaa76d47f3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:52:05.338995 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8c6e0bc3-ad43-446c-a6fb-9a697d5fa27f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:05.339620 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8c6e0bc3-ad43-446c-a6fb-9a697d5fa27f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:05.341105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1394/2791] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:52:05 2022] DELETE /identity/v3/projects/c2aea0eec9464f21b31cdcaa76d47f3b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:05.346033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d49928-ae1a-455a-bc23-34655cbf3d34 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:05.346528 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d49928-ae1a-455a-bc23-34655cbf3d34 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:05.346626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d49928-ae1a-455a-bc23-34655cbf3d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:05.367850 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f2d49928-ae1a-455a-bc23-34655cbf3d34 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:05.369093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f2d49928-ae1a-455a-bc23-34655cbf3d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:05.426268 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1398/2792] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:05.434815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5ae31-4ff0-47ac-8759-4f8b061a1bb1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:05.435097 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5ae31-4ff0-47ac-8759-4f8b061a1bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:05.435375 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5ae31-4ff0-47ac-8759-4f8b061a1bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:05.454907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-efd5ae31-4ff0-47ac-8759-4f8b061a1bb1 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:05.456109 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-efd5ae31-4ff0-47ac-8759-4f8b061a1bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:05.464375 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1395/2793] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:05.468794 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d49928-ae1a-455a-bc23-34655cbf3d34 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:05.479451 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d49928-ae1a-455a-bc23-34655cbf3d34 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:05.484303 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb87e0-c36b-4fbf-a80b-7ba49be28052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:05.484917 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb87e0-c36b-4fbf-a80b-7ba49be28052 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:05.485210 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb87e0-c36b-4fbf-a80b-7ba49be28052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:05.485486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb87e0-c36b-4fbf-a80b-7ba49be28052 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:05.486185 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb87e0-c36b-4fbf-a80b-7ba49be28052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:52:05.486725 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb87e0-c36b-4fbf-a80b-7ba49be28052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:05.488625 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb87e0-c36b-4fbf-a80b-7ba49be28052 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:05.502346 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1399/2794] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:52:05.506948 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5ae31-4ff0-47ac-8759-4f8b061a1bb1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:05.517607 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5ae31-4ff0-47ac-8759-4f8b061a1bb1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:05.522206 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a38c2-3173-47ed-9a90-319261448408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:05.522847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a38c2-3173-47ed-9a90-319261448408 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:05.523696 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a38c2-3173-47ed-9a90-319261448408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:05.523965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a38c2-3173-47ed-9a90-319261448408 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:05.524725 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a38c2-3173-47ed-9a90-319261448408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:05.526940 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a38c2-3173-47ed-9a90-319261448408 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:05.561154 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1396/2795] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:52:05 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:52:05.565993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb87e0-c36b-4fbf-a80b-7ba49be28052 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:05.576962 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb87e0-c36b-4fbf-a80b-7ba49be28052 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:05.599591 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:05.640123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c3d5c-9c39-418e-876f-fa0020eb7f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:05.640763 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c3d5c-9c39-418e-876f-fa0020eb7f21 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:05.641100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c3d5c-9c39-418e-876f-fa0020eb7f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:05.641359 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c3d5c-9c39-418e-876f-fa0020eb7f21 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:05.642066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c3d5c-9c39-418e-876f-fa0020eb7f21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:05.644020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c3d5c-9c39-418e-876f-fa0020eb7f21 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:05.666576 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1400/2796] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:52:05 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:52:05.670978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a38c2-3173-47ed-9a90-319261448408 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:05.681528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a38c2-3173-47ed-9a90-319261448408 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:05.686414 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752dc07-ec7d-4ab2-ad6b-2a423cbba01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:05.687026 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752dc07-ec7d-4ab2-ad6b-2a423cbba01a admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:05.687307 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752dc07-ec7d-4ab2-ad6b-2a423cbba01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:05.687581 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752dc07-ec7d-4ab2-ad6b-2a423cbba01a admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:05.688286 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b752dc07-ec7d-4ab2-ad6b-2a423cbba01a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:05.689938 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b752dc07-ec7d-4ab2-ad6b-2a423cbba01a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:05.700673 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1397/2797] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:52:05.704727 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c3d5c-9c39-418e-876f-fa0020eb7f21 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:05.715364 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c3d5c-9c39-418e-876f-fa0020eb7f21 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:05.720038 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6319ba0-f56b-4c22-8e92-ef2d6c1c7ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:05.720666 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6319ba0-f56b-4c22-8e92-ef2d6c1c7ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:05.720951 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6319ba0-f56b-4c22-8e92-ef2d6c1c7ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:05.721229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6319ba0-f56b-4c22-8e92-ef2d6c1c7ac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:05.721964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6319ba0-f56b-4c22-8e92-ef2d6c1c7ac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:05.723582 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6319ba0-f56b-4c22-8e92-ef2d6c1c7ac7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:05.735822 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1401/2798] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:52:05.740474 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752dc07-ec7d-4ab2-ad6b-2a423cbba01a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:05.751635 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752dc07-ec7d-4ab2-ad6b-2a423cbba01a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:05.756570 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c557c1-95ff-474d-88e9-f5b0c304171c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:05.757356 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c557c1-95ff-474d-88e9-f5b0c304171c admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:05.757581 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c557c1-95ff-474d-88e9-f5b0c304171c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:05.757858 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c557c1-95ff-474d-88e9-f5b0c304171c admin admin] PATH_INFO: `/v3/projects/d27f2a56dc8a40e2842db234f6f2f9df/users/1a64b70a7fdd471186295181fea23fea/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:05.760408 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:05.777142 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c557c1-95ff-474d-88e9-f5b0c304171c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d27f2a56dc8a40e2842db234f6f2f9df, user_id=1a64b70a7fdd471186295181fea23fea, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:05.779776 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c557c1-95ff-474d-88e9-f5b0c304171c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:05.795073 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1398/2799] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:52:05 2022] PUT /identity/v3/projects/d27f2a56dc8a40e2842db234f6f2f9df/users/1a64b70a7fdd471186295181fea23fea/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:05.806648 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dc5bb-747c-42ca-a084-03253826cdd0 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:05.806933 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dc5bb-747c-42ca-a084-03253826cdd0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:05.807211 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dc5bb-747c-42ca-a084-03253826cdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:05.837229 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-381dc5bb-747c-42ca-a084-03253826cdd0 None None] MFA Rules not processed for user `1a64b70a7fdd471186295181fea23fea`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:05.838513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-381dc5bb-747c-42ca-a084-03253826cdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:05.911433 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1402/2800] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:05.916215 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c557c1-95ff-474d-88e9-f5b0c304171c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:05.997873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c557c1-95ff-474d-88e9-f5b0c304171c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:06.005293 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f0101-090d-4baf-b6c6-9d0ae84b5bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:06.006428 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f0101-090d-4baf-b6c6-9d0ae84b5bed admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:06.006929 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f0101-090d-4baf-b6c6-9d0ae84b5bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:06.007931 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f0101-090d-4baf-b6c6-9d0ae84b5bed admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:06.009207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f0101-090d-4baf-b6c6-9d0ae84b5bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:06.012589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f0101-090d-4baf-b6c6-9d0ae84b5bed admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:06.053113 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1399/2801] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:52:05 2022] POST /identity/v3/projects => generated 377 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:52:06.057580 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381dc5bb-747c-42ca-a084-03253826cdd0 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:06.068559 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381dc5bb-747c-42ca-a084-03253826cdd0 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:06.129389 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793eea4-aeea-4389-81d9-2b439a127ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:06.130066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793eea4-aeea-4389-81d9-2b439a127ebf admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:06.130352 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793eea4-aeea-4389-81d9-2b439a127ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:06.130626 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793eea4-aeea-4389-81d9-2b439a127ebf admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:06.131343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793eea4-aeea-4389-81d9-2b439a127ebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:06.134079 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793eea4-aeea-4389-81d9-2b439a127ebf admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:06.159924 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1403/2802] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:52:06 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:52:06.164065 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f0101-090d-4baf-b6c6-9d0ae84b5bed admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:06.175173 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f0101-090d-4baf-b6c6-9d0ae84b5bed admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:06.180060 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357a076-e757-47d9-97a1-11699da27a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:06.180705 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357a076-e757-47d9-97a1-11699da27a9b admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:06.180996 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357a076-e757-47d9-97a1-11699da27a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:06.181264 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357a076-e757-47d9-97a1-11699da27a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:06.182012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357a076-e757-47d9-97a1-11699da27a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:06.183616 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357a076-e757-47d9-97a1-11699da27a9b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:06.196045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1400/2803] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:52:06.200311 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793eea4-aeea-4389-81d9-2b439a127ebf admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:06.212020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793eea4-aeea-4389-81d9-2b439a127ebf admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:06.217045 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17124b5-c8b2-4e28-8026-618afc51c574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:06.217720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17124b5-c8b2-4e28-8026-618afc51c574 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:06.217999 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17124b5-c8b2-4e28-8026-618afc51c574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:06.218271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17124b5-c8b2-4e28-8026-618afc51c574 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:06.218973 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17124b5-c8b2-4e28-8026-618afc51c574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:06.220640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17124b5-c8b2-4e28-8026-618afc51c574 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:06.231579 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1404/2804] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:52:06.235947 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357a076-e757-47d9-97a1-11699da27a9b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:06.247412 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357a076-e757-47d9-97a1-11699da27a9b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:06.252316 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa73046-8b4f-4857-9ce7-e4f7b518a7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:06.252998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa73046-8b4f-4857-9ce7-e4f7b518a7bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:06.253277 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa73046-8b4f-4857-9ce7-e4f7b518a7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:06.253582 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa73046-8b4f-4857-9ce7-e4f7b518a7bc admin admin] PATH_INFO: `/v3/projects/005a89c14a054c89b5ffbc631142b309/users/b12dc1c8148c4735879fe7f094006db0/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:06.269845 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa73046-8b4f-4857-9ce7-e4f7b518a7bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=005a89c14a054c89b5ffbc631142b309, user_id=b12dc1c8148c4735879fe7f094006db0, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:06.272487 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa73046-8b4f-4857-9ce7-e4f7b518a7bc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:06.288099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1401/2805] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:52:06 2022] PUT /identity/v3/projects/005a89c14a054c89b5ffbc631142b309/users/b12dc1c8148c4735879fe7f094006db0/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:06.299533 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e337c0b-bc26-4526-868c-15084606a71d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:06.299793 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e337c0b-bc26-4526-868c-15084606a71d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:06.300067 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e337c0b-bc26-4526-868c-15084606a71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:06.323042 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:06.338961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7e337c0b-bc26-4526-868c-15084606a71d None None] MFA Rules not processed for user `b12dc1c8148c4735879fe7f094006db0`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:06.340392 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7e337c0b-bc26-4526-868c-15084606a71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:06.415566 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1405/2806] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:06.420234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa73046-8b4f-4857-9ce7-e4f7b518a7bc admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:06.490020 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa73046-8b4f-4857-9ce7-e4f7b518a7bc admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:06.495549 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62054a-fa2a-4c72-bfff-4688081b7517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:06.496186 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62054a-fa2a-4c72-bfff-4688081b7517 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:06.496468 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62054a-fa2a-4c72-bfff-4688081b7517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:06.496743 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62054a-fa2a-4c72-bfff-4688081b7517 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:06.497431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62054a-fa2a-4c72-bfff-4688081b7517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:06.499361 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62054a-fa2a-4c72-bfff-4688081b7517 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:06.531232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1402/2807] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:52:06 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:52:06.535723 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e337c0b-bc26-4526-868c-15084606a71d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:06.546898 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e337c0b-bc26-4526-868c-15084606a71d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:06.627312 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470dd57-2f4d-4603-94c4-714e3477e47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:06.627873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470dd57-2f4d-4603-94c4-714e3477e47d admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:06.628190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470dd57-2f4d-4603-94c4-714e3477e47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:06.628472 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470dd57-2f4d-4603-94c4-714e3477e47d admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:06.629173 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470dd57-2f4d-4603-94c4-714e3477e47d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:06.631176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470dd57-2f4d-4603-94c4-714e3477e47d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:06.666482 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1406/2808] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:52:06 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:52:06.671357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62054a-fa2a-4c72-bfff-4688081b7517 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:06.684014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62054a-fa2a-4c72-bfff-4688081b7517 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:06.692123 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e570adc-3f43-497c-84de-42b33faaf1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:06.692353 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e570adc-3f43-497c-84de-42b33faaf1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:06.692685 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e570adc-3f43-497c-84de-42b33faaf1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:06.693061 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e570adc-3f43-497c-84de-42b33faaf1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:06.693914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e570adc-3f43-497c-84de-42b33faaf1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:06.695651 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e570adc-3f43-497c-84de-42b33faaf1a7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:06.707924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1403/2809] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 10:52:06.712425 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470dd57-2f4d-4603-94c4-714e3477e47d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:06.722965 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470dd57-2f4d-4603-94c4-714e3477e47d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:06.729701 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae14d0-705d-450d-8162-25d71ba9760b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:06.730784 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae14d0-705d-450d-8162-25d71ba9760b admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:06.731153 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae14d0-705d-450d-8162-25d71ba9760b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:06.731597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae14d0-705d-450d-8162-25d71ba9760b admin admin] PATH_INFO: `/v3/projects/e39d15093de54dffa0f2d2d909cf55d9/users/abb5886fe5044452940e72be4972e8e3/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:06.750689 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae14d0-705d-450d-8162-25d71ba9760b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e39d15093de54dffa0f2d2d909cf55d9, user_id=abb5886fe5044452940e72be4972e8e3, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:06.753367 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae14d0-705d-450d-8162-25d71ba9760b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:06.769020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1407/2810] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:52:06 2022] PUT /identity/v3/projects/e39d15093de54dffa0f2d2d909cf55d9/users/abb5886fe5044452940e72be4972e8e3/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:06.773886 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e570adc-3f43-497c-84de-42b33faaf1a7 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:06.786405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e570adc-3f43-497c-84de-42b33faaf1a7 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:06.790287 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:06.847815 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85125f67-b511-4ba8-b541-4bc9e3c48939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:06.848518 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85125f67-b511-4ba8-b541-4bc9e3c48939 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:06.848856 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85125f67-b511-4ba8-b541-4bc9e3c48939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:06.849211 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85125f67-b511-4ba8-b541-4bc9e3c48939 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:06.850043 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85125f67-b511-4ba8-b541-4bc9e3c48939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:06.851709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85125f67-b511-4ba8-b541-4bc9e3c48939 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:06.862642 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1404/2811] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:52:06.867742 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae14d0-705d-450d-8162-25d71ba9760b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:06.878684 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae14d0-705d-450d-8162-25d71ba9760b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:06.883913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7871a-7620-437c-8cdb-709590dbce3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:06.884614 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7871a-7620-437c-8cdb-709590dbce3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:06.884891 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7871a-7620-437c-8cdb-709590dbce3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:06.885169 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7871a-7620-437c-8cdb-709590dbce3c admin admin] PATH_INFO: `/v3/projects/e39d15093de54dffa0f2d2d909cf55d9/users/abb5886fe5044452940e72be4972e8e3/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:06.887233 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7871a-7620-437c-8cdb-709590dbce3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e39d15093de54dffa0f2d2d909cf55d9, user_id=abb5886fe5044452940e72be4972e8e3, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:06.889932 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7871a-7620-437c-8cdb-709590dbce3c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:06.910805 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1408/2812] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:52:06 2022] PUT /identity/v3/projects/e39d15093de54dffa0f2d2d909cf55d9/users/abb5886fe5044452940e72be4972e8e3/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:06.923368 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3779d5-155b-437f-a32c-180ec6960052 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:06.923655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3779d5-155b-437f-a32c-180ec6960052 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:06.923964 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3779d5-155b-437f-a32c-180ec6960052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:06.954133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3b3779d5-155b-437f-a32c-180ec6960052 None None] MFA Rules not processed for user `abb5886fe5044452940e72be4972e8e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:06.955374 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3b3779d5-155b-437f-a32c-180ec6960052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:07.028899 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1405/2813] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:07.041020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7871a-7620-437c-8cdb-709590dbce3c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:07.043170 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-78a7871a-7620-437c-8cdb-709590dbce3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:07.102410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7871a-7620-437c-8cdb-709590dbce3c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:07.106066 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466f299-4830-430a-a4a2-2f8afe950bbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:07.106695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466f299-4830-430a-a4a2-2f8afe950bbc service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:07.106978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466f299-4830-430a-a4a2-2f8afe950bbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:07.107254 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466f299-4830-430a-a4a2-2f8afe950bbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:07.111657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:07.174989 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466f299-4830-430a-a4a2-2f8afe950bbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:07.176338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466f299-4830-430a-a4a2-2f8afe950bbc service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:07.186123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1409/2814] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:52:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:52:08.727829 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3779d5-155b-437f-a32c-180ec6960052 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:08.729231 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3b3779d5-155b-437f-a32c-180ec6960052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:08.741575 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3779d5-155b-437f-a32c-180ec6960052 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:08.745707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3ace8-ee7a-44f4-8b48-39a981d7edd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:08.746336 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3ace8-ee7a-44f4-8b48-39a981d7edd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:08.746606 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3ace8-ee7a-44f4-8b48-39a981d7edd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:08.746880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3ace8-ee7a-44f4-8b48-39a981d7edd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:08.809171 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3ace8-ee7a-44f4-8b48-39a981d7edd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:08.810626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3ace8-ee7a-44f4-8b48-39a981d7edd3 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:08.821979 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1406/2815] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:52:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:52:09.601105 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466f299-4830-430a-a4a2-2f8afe950bbc service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:09.610694 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466f299-4830-430a-a4a2-2f8afe950bbc service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:09.614471 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83dc450-e13e-47c1-ae0a-9041b366067d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:09.615101 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dc450-e13e-47c1-ae0a-9041b366067d service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:09.615382 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dc450-e13e-47c1-ae0a-9041b366067d service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:09.615659 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dc450-e13e-47c1-ae0a-9041b366067d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:09.625771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83dc450-e13e-47c1-ae0a-9041b366067d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:09.627255 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83dc450-e13e-47c1-ae0a-9041b366067d service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:09.638074 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1410/2816] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:52:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:52:13.677479 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3ace8-ee7a-44f4-8b48-39a981d7edd3 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:13.709197 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:13.745115 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3ace8-ee7a-44f4-8b48-39a981d7edd3 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:13.749675 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792846bb-56eb-4cd7-968c-323bdf3012f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:13.750325 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792846bb-56eb-4cd7-968c-323bdf3012f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:13.750602 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792846bb-56eb-4cd7-968c-323bdf3012f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:13.750872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792846bb-56eb-4cd7-968c-323bdf3012f5 admin admin] PATH_INFO: `/v3/users/1a64b70a7fdd471186295181fea23fea` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:13.752171 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792846bb-56eb-4cd7-968c-323bdf3012f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a64b70a7fdd471186295181fea23fea)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:13.754547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792846bb-56eb-4cd7-968c-323bdf3012f5 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:13.803676 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-792846bb-56eb-4cd7-968c-323bdf3012f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a64b70a7fdd471186295181fea23fea'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:13.815660 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-792846bb-56eb-4cd7-968c-323bdf3012f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a64b70a7fdd471186295181fea23fea'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:13.816305 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-792846bb-56eb-4cd7-968c-323bdf3012f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a64b70a7fdd471186295181fea23fea'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:13.824535 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-792846bb-56eb-4cd7-968c-323bdf3012f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a64b70a7fdd471186295181fea23fea'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:13.831692 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-792846bb-56eb-4cd7-968c-323bdf3012f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a64b70a7fdd471186295181fea23fea'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:13.838529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1407/2817] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:52:13 2022] DELETE /identity/v3/users/1a64b70a7fdd471186295181fea23fea => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:13.843274 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d2106-4394-47fa-afd0-852f28603247 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:13.844030 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d2106-4394-47fa-afd0-852f28603247 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:13.844030 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d2106-4394-47fa-afd0-852f28603247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:13.863404 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d16d2106-4394-47fa-afd0-852f28603247 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:13.864627 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d16d2106-4394-47fa-afd0-852f28603247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:13.926661 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1411/2818] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:52:13.938495 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792846bb-56eb-4cd7-968c-323bdf3012f5 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:13.940010 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-792846bb-56eb-4cd7-968c-323bdf3012f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:14.011632 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792846bb-56eb-4cd7-968c-323bdf3012f5 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:14.015891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df64338b-0624-4aab-89c3-fcabe46053a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:14.016525 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64338b-0624-4aab-89c3-fcabe46053a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:14.016806 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64338b-0624-4aab-89c3-fcabe46053a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:14.017092 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64338b-0624-4aab-89c3-fcabe46053a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:14.027702 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df64338b-0624-4aab-89c3-fcabe46053a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:14.029165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df64338b-0624-4aab-89c3-fcabe46053a2 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:14.040699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1408/2819] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:52:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:52:14.347162 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d2106-4394-47fa-afd0-852f28603247 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:14.349411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d16d2106-4394-47fa-afd0-852f28603247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:14.376177 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d2106-4394-47fa-afd0-852f28603247 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:14.384397 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:14.385582 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:14.386099 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:14.386621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c admin admin] PATH_INFO: `/v3/projects/d27f2a56dc8a40e2842db234f6f2f9df` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:14.388856 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d27f2a56dc8a40e2842db234f6f2f9df)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:14.392702 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:14.463733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd27f2a56dc8a40e2842db234f6f2f9df', 'request_id': 'req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:14.470532 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd27f2a56dc8a40e2842db234f6f2f9df', 'request_id': 'req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:14.478949 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd27f2a56dc8a40e2842db234f6f2f9df', 'request_id': 'req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:14.490344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c admin admin] The token cache is being invalidate because project d27f2a56dc8a40e2842db234f6f2f9df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:52:14.490711 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:14.492429 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:14.494649 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1412/2820] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:52:14 2022] DELETE /identity/v3/projects/d27f2a56dc8a40e2842db234f6f2f9df => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:14.499144 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df64338b-0624-4aab-89c3-fcabe46053a2 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:14.510306 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df64338b-0624-4aab-89c3-fcabe46053a2 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:14.565873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:14.580241 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0640c-29cd-441d-b150-0491c02f7106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:14.580933 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0640c-29cd-441d-b150-0491c02f7106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:14.581240 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0640c-29cd-441d-b150-0491c02f7106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:14.581548 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0640c-29cd-441d-b150-0491c02f7106 admin admin] PATH_INFO: `/v3/users/b12dc1c8148c4735879fe7f094006db0` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:14.582820 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0640c-29cd-441d-b150-0491c02f7106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12dc1c8148c4735879fe7f094006db0)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:14.584761 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0640c-29cd-441d-b150-0491c02f7106 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:14.637565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e2a0640c-29cd-441d-b150-0491c02f7106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12dc1c8148c4735879fe7f094006db0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:14.658917 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e2a0640c-29cd-441d-b150-0491c02f7106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12dc1c8148c4735879fe7f094006db0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:14.659664 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e2a0640c-29cd-441d-b150-0491c02f7106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12dc1c8148c4735879fe7f094006db0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:14.668446 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e2a0640c-29cd-441d-b150-0491c02f7106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12dc1c8148c4735879fe7f094006db0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:14.676662 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e2a0640c-29cd-441d-b150-0491c02f7106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12dc1c8148c4735879fe7f094006db0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:14.683470 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1409/2821] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:52:14 2022] DELETE /identity/v3/users/b12dc1c8148c4735879fe7f094006db0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:52:15.126558 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:15.128231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:15.143915 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81a0a6-1f86-4a2c-8fa3-e72197fe0f2c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:15.159812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:15.219764 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c524d-7146-4b52-8ba6-f91319c764f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:15.220432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c524d-7146-4b52-8ba6-f91319c764f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:15.220710 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c524d-7146-4b52-8ba6-f91319c764f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:15.220988 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c524d-7146-4b52-8ba6-f91319c764f8 admin admin] PATH_INFO: `/v3/projects/005a89c14a054c89b5ffbc631142b309` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:15.222890 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c524d-7146-4b52-8ba6-f91319c764f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=005a89c14a054c89b5ffbc631142b309)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:15.224826 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c524d-7146-4b52-8ba6-f91319c764f8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:15.278199 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e71c524d-7146-4b52-8ba6-f91319c764f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '005a89c14a054c89b5ffbc631142b309', 'request_id': 'req-e71c524d-7146-4b52-8ba6-f91319c764f8'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:15.285093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e71c524d-7146-4b52-8ba6-f91319c764f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '005a89c14a054c89b5ffbc631142b309', 'request_id': 'req-e71c524d-7146-4b52-8ba6-f91319c764f8'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:15.294444 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e71c524d-7146-4b52-8ba6-f91319c764f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '005a89c14a054c89b5ffbc631142b309', 'request_id': 'req-e71c524d-7146-4b52-8ba6-f91319c764f8'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:15.306620 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e71c524d-7146-4b52-8ba6-f91319c764f8 admin admin] The token cache is being invalidate because project 005a89c14a054c89b5ffbc631142b309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:52:15.306950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e71c524d-7146-4b52-8ba6-f91319c764f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:15.307572 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e71c524d-7146-4b52-8ba6-f91319c764f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:15.309442 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1413/2822] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:52:15 2022] DELETE /identity/v3/projects/005a89c14a054c89b5ffbc631142b309 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:52:15.313636 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0640c-29cd-441d-b150-0491c02f7106 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:15.315266 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e2a0640c-29cd-441d-b150-0491c02f7106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:15.329091 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0640c-29cd-441d-b150-0491c02f7106 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:15.400813 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39019bfa-1eee-4080-826e-251a76261668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:15.401491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39019bfa-1eee-4080-826e-251a76261668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:15.401834 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39019bfa-1eee-4080-826e-251a76261668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:15.402081 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39019bfa-1eee-4080-826e-251a76261668 admin admin] PATH_INFO: `/v3/users/abb5886fe5044452940e72be4972e8e3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:15.403350 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39019bfa-1eee-4080-826e-251a76261668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abb5886fe5044452940e72be4972e8e3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:15.405636 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39019bfa-1eee-4080-826e-251a76261668 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:15.465670 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-39019bfa-1eee-4080-826e-251a76261668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abb5886fe5044452940e72be4972e8e3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:15.480098 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-39019bfa-1eee-4080-826e-251a76261668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abb5886fe5044452940e72be4972e8e3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:15.480771 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-39019bfa-1eee-4080-826e-251a76261668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abb5886fe5044452940e72be4972e8e3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:15.484868 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:15.493821 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-39019bfa-1eee-4080-826e-251a76261668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abb5886fe5044452940e72be4972e8e3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:15.504547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-39019bfa-1eee-4080-826e-251a76261668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abb5886fe5044452940e72be4972e8e3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:15.511817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1410/2823] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:52:15 2022] DELETE /identity/v3/users/abb5886fe5044452940e72be4972e8e3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:15.793819 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c524d-7146-4b52-8ba6-f91319c764f8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:15.795241 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e71c524d-7146-4b52-8ba6-f91319c764f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:15.808609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c524d-7146-4b52-8ba6-f91319c764f8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:15.867417 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ba3f0-33eb-45cb-9c2f-42b30bad6f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:15.868100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ba3f0-33eb-45cb-9c2f-42b30bad6f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:15.868380 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ba3f0-33eb-45cb-9c2f-42b30bad6f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:15.868653 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ba3f0-33eb-45cb-9c2f-42b30bad6f54 admin admin] PATH_INFO: `/v3/projects/e39d15093de54dffa0f2d2d909cf55d9` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:15.869842 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ba3f0-33eb-45cb-9c2f-42b30bad6f54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e39d15093de54dffa0f2d2d909cf55d9)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:15.871767 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ba3f0-33eb-45cb-9c2f-42b30bad6f54 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:15.916904 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:15.919172 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e98ba3f0-33eb-45cb-9c2f-42b30bad6f54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e39d15093de54dffa0f2d2d909cf55d9', 'request_id': 'req-e98ba3f0-33eb-45cb-9c2f-42b30bad6f54'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:15.928241 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e98ba3f0-33eb-45cb-9c2f-42b30bad6f54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e39d15093de54dffa0f2d2d909cf55d9', 'request_id': 'req-e98ba3f0-33eb-45cb-9c2f-42b30bad6f54'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:15.937126 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e98ba3f0-33eb-45cb-9c2f-42b30bad6f54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e39d15093de54dffa0f2d2d909cf55d9', 'request_id': 'req-e98ba3f0-33eb-45cb-9c2f-42b30bad6f54'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:15.947455 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e98ba3f0-33eb-45cb-9c2f-42b30bad6f54 admin admin] The token cache is being invalidate because project e39d15093de54dffa0f2d2d909cf55d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:52:15.947851 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e98ba3f0-33eb-45cb-9c2f-42b30bad6f54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:15.948757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e98ba3f0-33eb-45cb-9c2f-42b30bad6f54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:52:15.950296 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1414/2824] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:52:15 2022] DELETE /identity/v3/projects/e39d15093de54dffa0f2d2d909cf55d9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:52:15.955363 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a918700-ca1a-41ec-b998-383b70a29791 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:15.955873 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a918700-ca1a-41ec-b998-383b70a29791 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:15.955983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a918700-ca1a-41ec-b998-383b70a29791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:15.976437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0a918700-ca1a-41ec-b998-383b70a29791 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:15.977691 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0a918700-ca1a-41ec-b998-383b70a29791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:16.041604 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1411/2825] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:16.051178 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877273e-3649-41cf-b184-bda9d539171c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:16.051474 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877273e-3649-41cf-b184-bda9d539171c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:16.051757 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877273e-3649-41cf-b184-bda9d539171c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:16.071354 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5877273e-3649-41cf-b184-bda9d539171c None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:16.072632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5877273e-3649-41cf-b184-bda9d539171c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:16.083737 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1415/2826] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:52:16.093081 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a918700-ca1a-41ec-b998-383b70a29791 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:16.103755 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a918700-ca1a-41ec-b998-383b70a29791 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:16.108589 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190df7d-6758-4a70-b5af-05daa9567206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:16.109263 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190df7d-6758-4a70-b5af-05daa9567206 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:16.109558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190df7d-6758-4a70-b5af-05daa9567206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:16.110105 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190df7d-6758-4a70-b5af-05daa9567206 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:16.110568 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190df7d-6758-4a70-b5af-05daa9567206 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:52:16.110911 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190df7d-6758-4a70-b5af-05daa9567206 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:16.112514 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190df7d-6758-4a70-b5af-05daa9567206 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:16.129164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1412/2827] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:52:16.130790 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877273e-3649-41cf-b184-bda9d539171c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:16.140885 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877273e-3649-41cf-b184-bda9d539171c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:16.145720 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e67be-0524-4b14-94a9-d5b56bcccdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:16.146347 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e67be-0524-4b14-94a9-d5b56bcccdff admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:16.146628 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e67be-0524-4b14-94a9-d5b56bcccdff admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:16.146902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e67be-0524-4b14-94a9-d5b56bcccdff admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:16.147621 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e67be-0524-4b14-94a9-d5b56bcccdff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:16.150137 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e67be-0524-4b14-94a9-d5b56bcccdff admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:16.179059 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1416/2828] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:52:16 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:52:16.183288 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190df7d-6758-4a70-b5af-05daa9567206 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:16.193245 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190df7d-6758-4a70-b5af-05daa9567206 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:16.251529 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10026ec5-57f1-46e8-90f2-86b3c9747efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:16.252069 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10026ec5-57f1-46e8-90f2-86b3c9747efd admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:16.252347 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10026ec5-57f1-46e8-90f2-86b3c9747efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:16.252622 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10026ec5-57f1-46e8-90f2-86b3c9747efd admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:16.253293 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10026ec5-57f1-46e8-90f2-86b3c9747efd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:16.255216 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10026ec5-57f1-46e8-90f2-86b3c9747efd admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:16.276591 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1413/2829] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:52:16 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:52:16.280733 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e67be-0524-4b14-94a9-d5b56bcccdff admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:16.292245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e67be-0524-4b14-94a9-d5b56bcccdff admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:16.297056 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff95afd-27c5-4767-9cfb-ed024adfe565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:16.297705 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff95afd-27c5-4767-9cfb-ed024adfe565 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:16.297982 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff95afd-27c5-4767-9cfb-ed024adfe565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:16.298253 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff95afd-27c5-4767-9cfb-ed024adfe565 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:16.298938 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff95afd-27c5-4767-9cfb-ed024adfe565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:16.300513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff95afd-27c5-4767-9cfb-ed024adfe565 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:16.310923 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1417/2830] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:52:16.315048 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10026ec5-57f1-46e8-90f2-86b3c9747efd admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:16.325234 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10026ec5-57f1-46e8-90f2-86b3c9747efd admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:16.329990 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea791bb-4838-4c31-99fb-308da1f43e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:16.330610 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea791bb-4838-4c31-99fb-308da1f43e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:16.330894 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea791bb-4838-4c31-99fb-308da1f43e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:16.331188 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea791bb-4838-4c31-99fb-308da1f43e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:16.331891 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea791bb-4838-4c31-99fb-308da1f43e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:16.333493 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea791bb-4838-4c31-99fb-308da1f43e12 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:16.345087 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1414/2831] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:52:16.349327 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff95afd-27c5-4767-9cfb-ed024adfe565 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:16.360104 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff95afd-27c5-4767-9cfb-ed024adfe565 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:16.364993 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461068b-8b25-4fd9-8fb9-f3a5ed31cdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:16.365783 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461068b-8b25-4fd9-8fb9-f3a5ed31cdf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:16.366037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461068b-8b25-4fd9-8fb9-f3a5ed31cdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:16.366540 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461068b-8b25-4fd9-8fb9-f3a5ed31cdf6 admin admin] PATH_INFO: `/v3/projects/f8c75fcb21364890945aead8f411c406/users/fe00592df00740cea113a9bfc2d71e76/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:16.382760 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461068b-8b25-4fd9-8fb9-f3a5ed31cdf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c75fcb21364890945aead8f411c406, user_id=fe00592df00740cea113a9bfc2d71e76, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:16.385374 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461068b-8b25-4fd9-8fb9-f3a5ed31cdf6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:16.399538 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1418/2832] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:52:16 2022] PUT /identity/v3/projects/f8c75fcb21364890945aead8f411c406/users/fe00592df00740cea113a9bfc2d71e76/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 10:52:16.404715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b29637-2e79-4e0f-b6a5-90cdcf5e2a69 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:16.405029 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b29637-2e79-4e0f-b6a5-90cdcf5e2a69 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:16.405313 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b29637-2e79-4e0f-b6a5-90cdcf5e2a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:16.425391 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a0b29637-2e79-4e0f-b6a5-90cdcf5e2a69 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:16.426635 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a0b29637-2e79-4e0f-b6a5-90cdcf5e2a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:16.479492 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:16.493581 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1415/2833] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:16.503473 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461068b-8b25-4fd9-8fb9-f3a5ed31cdf6 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:16.505163 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0461068b-8b25-4fd9-8fb9-f3a5ed31cdf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:16.565159 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461068b-8b25-4fd9-8fb9-f3a5ed31cdf6 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:16.569057 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11f70f-347d-40f7-a02a-aa79b9bd20df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:16.570262 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11f70f-347d-40f7-a02a-aa79b9bd20df service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:16.570552 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11f70f-347d-40f7-a02a-aa79b9bd20df service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:16.570831 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11f70f-347d-40f7-a02a-aa79b9bd20df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:16.581047 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b11f70f-347d-40f7-a02a-aa79b9bd20df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:16.582567 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b11f70f-347d-40f7-a02a-aa79b9bd20df service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:16.593823 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1419/2834] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:52:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:52:17.283385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f930e0-2acf-40e7-b603-a4a001f94d05 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:17.284045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f930e0-2acf-40e7-b603-a4a001f94d05 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:17.284045 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f930e0-2acf-40e7-b603-a4a001f94d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:17.315513 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-19f930e0-2acf-40e7-b603-a4a001f94d05 None None] MFA Rules not processed for user `fe00592df00740cea113a9bfc2d71e76`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:17.316902 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-19f930e0-2acf-40e7-b603-a4a001f94d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:17.398092 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1416/2835] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:17.409496 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11f70f-347d-40f7-a02a-aa79b9bd20df service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:17.411071 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5b11f70f-347d-40f7-a02a-aa79b9bd20df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:17.471521 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11f70f-347d-40f7-a02a-aa79b9bd20df service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:17.475326 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a94aa-eed0-4eba-badd-a3b3e3204850 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:17.476000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a94aa-eed0-4eba-badd-a3b3e3204850 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:17.476269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a94aa-eed0-4eba-badd-a3b3e3204850 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:17.476545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a94aa-eed0-4eba-badd-a3b3e3204850 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:17.485260 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a94aa-eed0-4eba-badd-a3b3e3204850 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:17.486738 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a94aa-eed0-4eba-badd-a3b3e3204850 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:17.496294 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1420/2836] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:52:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:52:18.879417 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660b29e-beda-48e3-ac7f-d0ec1442559b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:18.879973 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660b29e-beda-48e3-ac7f-d0ec1442559b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:18.879973 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660b29e-beda-48e3-ac7f-d0ec1442559b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:18.898482 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c660b29e-beda-48e3-ac7f-d0ec1442559b None None] MFA Rules not processed for user `fe00592df00740cea113a9bfc2d71e76`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:18.899686 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c660b29e-beda-48e3-ac7f-d0ec1442559b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:18.906547 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1417/2837] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:27.492016 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:28.893432 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=38 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:33.219245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ff4e7-bfdb-469c-b5a0-57968ebb4e3c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:33.219735 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ff4e7-bfdb-469c-b5a0-57968ebb4e3c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:33.219838 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ff4e7-bfdb-469c-b5a0-57968ebb4e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:33.239323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-578ff4e7-bfdb-469c-b5a0-57968ebb4e3c None None] MFA Rules not processed for user `fe00592df00740cea113a9bfc2d71e76`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:33.240537 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-578ff4e7-bfdb-469c-b5a0-57968ebb4e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:33.247678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1421/2838] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:52:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:43.233682 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:52:54.079729 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a235e7b-f7c9-48db-b9ac-8f0ab6e12446 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:54.080539 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a235e7b-f7c9-48db-b9ac-8f0ab6e12446 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:54.080715 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a235e7b-f7c9-48db-b9ac-8f0ab6e12446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:54.111566 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7a235e7b-f7c9-48db-b9ac-8f0ab6e12446 None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:52:54.112816 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7a235e7b-f7c9-48db-b9ac-8f0ab6e12446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:54.170387 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1418/2839] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 3 10:52:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:52:54.213978 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ff4e7-bfdb-469c-b5a0-57968ebb4e3c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:52:54.215462 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-578ff4e7-bfdb-469c-b5a0-57968ebb4e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:52:54.290448 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ff4e7-bfdb-469c-b5a0-57968ebb4e3c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:52:54.295028 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862da15-adcb-406c-972f-2b63f2abd9d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:52:54.295676 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862da15-adcb-406c-972f-2b63f2abd9d1 service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:52:54.296275 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862da15-adcb-406c-972f-2b63f2abd9d1 service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:52:54.296275 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862da15-adcb-406c-972f-2b63f2abd9d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:52:54.306709 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862da15-adcb-406c-972f-2b63f2abd9d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:52:54.308459 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862da15-adcb-406c-972f-2b63f2abd9d1 service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:52:54.317913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1422/2840] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:52:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 10:53:04.163329 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=29 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:53:04.314087 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=32 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:53:11.050063 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34600d-c34a-4271-8cbd-4639c4010ecb None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:11.050685 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34600d-c34a-4271-8cbd-4639c4010ecb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:11.050958 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34600d-c34a-4271-8cbd-4639c4010ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:11.072645 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5f34600d-c34a-4271-8cbd-4639c4010ecb None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:53:11.073950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5f34600d-c34a-4271-8cbd-4639c4010ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:11.081666 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1419/2841] 10.222.0.47 () {58 vars in 967 bytes} [Wed Aug 3 10:53:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:53:11.094895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862da15-adcb-406c-972f-2b63f2abd9d1 service swift] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:11.106947 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862da15-adcb-406c-972f-2b63f2abd9d1 service swift] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:11.112184 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20ff5e-77b6-414b-9978-3134d7473b42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:11.112814 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20ff5e-77b6-414b-9978-3134d7473b42 service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:11.113094 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20ff5e-77b6-414b-9978-3134d7473b42 service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:11.113372 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20ff5e-77b6-414b-9978-3134d7473b42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:11.122400 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20ff5e-77b6-414b-9978-3134d7473b42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:11.123984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20ff5e-77b6-414b-9978-3134d7473b42 service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:11.133824 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1423/2842] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:53:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 10:53:12.938836 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de7597-a59c-4988-bfe7-98bfdb49a043 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:12.939369 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de7597-a59c-4988-bfe7-98bfdb49a043 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:12.939369 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de7597-a59c-4988-bfe7-98bfdb49a043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:12.959426 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d8de7597-a59c-4988-bfe7-98bfdb49a043 None None] MFA Rules not processed for user `035ae78ef3cb43e985e06cc3fa971c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:53:12.960752 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d8de7597-a59c-4988-bfe7-98bfdb49a043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:12.970504 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1420/2843] 10.222.0.47 () {58 vars in 967 bytes} [Wed Aug 3 10:53:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:53:12.983986 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20ff5e-77b6-414b-9978-3134d7473b42 service swift] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:12.998484 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20ff5e-77b6-414b-9978-3134d7473b42 service swift] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:13.003841 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e86ee-9574-477f-9ed2-32fb0f8636f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81ea5afe74b43879abc94b3cd134fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:13.004484 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e86ee-9574-477f-9ed2-32fb0f8636f5 service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:13.004774 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e86ee-9574-477f-9ed2-32fb0f8636f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:13.005056 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e86ee-9574-477f-9ed2-32fb0f8636f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:13.014808 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e86ee-9574-477f-9ed2-32fb0f8636f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:13.016457 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e86ee-9574-477f-9ed2-32fb0f8636f5 service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:13.026000 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1424/2844] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 3 10:53:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 10:53:13.250183 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de7597-a59c-4988-bfe7-98bfdb49a043 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:13.266372 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de7597-a59c-4988-bfe7-98bfdb49a043 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:13.273200 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b553707-9308-4273-b1a4-a2cee2b42a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:13.273961 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b553707-9308-4273-b1a4-a2cee2b42a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:13.274262 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b553707-9308-4273-b1a4-a2cee2b42a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:13.274548 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b553707-9308-4273-b1a4-a2cee2b42a0f admin admin] PATH_INFO: `/v3/users/fe00592df00740cea113a9bfc2d71e76` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:13.275830 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b553707-9308-4273-b1a4-a2cee2b42a0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe00592df00740cea113a9bfc2d71e76)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:13.277831 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b553707-9308-4273-b1a4-a2cee2b42a0f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:13.349595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9b553707-9308-4273-b1a4-a2cee2b42a0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe00592df00740cea113a9bfc2d71e76'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:13.366150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9b553707-9308-4273-b1a4-a2cee2b42a0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe00592df00740cea113a9bfc2d71e76'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:13.367024 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9b553707-9308-4273-b1a4-a2cee2b42a0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe00592df00740cea113a9bfc2d71e76'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:13.376769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9b553707-9308-4273-b1a4-a2cee2b42a0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe00592df00740cea113a9bfc2d71e76'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:13.385246 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9b553707-9308-4273-b1a4-a2cee2b42a0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe00592df00740cea113a9bfc2d71e76'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:13.392506 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1421/2845] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:53:13 2022] DELETE /identity/v3/users/fe00592df00740cea113a9bfc2d71e76 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:53:14.004299 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e86ee-9574-477f-9ed2-32fb0f8636f5 service swift] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:14.019542 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e86ee-9574-477f-9ed2-32fb0f8636f5 service swift] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:14.085916 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093fdf35-5b5c-4e8d-aef6-43c8dd3946fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:14.087324 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093fdf35-5b5c-4e8d-aef6-43c8dd3946fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:14.087475 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093fdf35-5b5c-4e8d-aef6-43c8dd3946fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:14.087734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093fdf35-5b5c-4e8d-aef6-43c8dd3946fc admin admin] PATH_INFO: `/v3/projects/f8c75fcb21364890945aead8f411c406` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:14.088895 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093fdf35-5b5c-4e8d-aef6-43c8dd3946fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8c75fcb21364890945aead8f411c406)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:14.090996 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093fdf35-5b5c-4e8d-aef6-43c8dd3946fc admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:14.139115 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-093fdf35-5b5c-4e8d-aef6-43c8dd3946fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8c75fcb21364890945aead8f411c406', 'request_id': 'req-093fdf35-5b5c-4e8d-aef6-43c8dd3946fc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:14.145360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-093fdf35-5b5c-4e8d-aef6-43c8dd3946fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8c75fcb21364890945aead8f411c406', 'request_id': 'req-093fdf35-5b5c-4e8d-aef6-43c8dd3946fc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:14.155658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-093fdf35-5b5c-4e8d-aef6-43c8dd3946fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8c75fcb21364890945aead8f411c406', 'request_id': 'req-093fdf35-5b5c-4e8d-aef6-43c8dd3946fc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:14.166609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-093fdf35-5b5c-4e8d-aef6-43c8dd3946fc admin admin] The token cache is being invalidate because project f8c75fcb21364890945aead8f411c406 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:53:14.166902 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-093fdf35-5b5c-4e8d-aef6-43c8dd3946fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:14.167545 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-093fdf35-5b5c-4e8d-aef6-43c8dd3946fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:14.169294 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1425/2846] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:53:14 2022] DELETE /identity/v3/projects/f8c75fcb21364890945aead8f411c406 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:53:14.176936 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80277d64-96de-4b76-a9f4-8bb98fec4de4 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:14.177439 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80277d64-96de-4b76-a9f4-8bb98fec4de4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:14.177650 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80277d64-96de-4b76-a9f4-8bb98fec4de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:14.201855 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-80277d64-96de-4b76-a9f4-8bb98fec4de4 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:53:14.203076 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-80277d64-96de-4b76-a9f4-8bb98fec4de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:14.269441 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1422/2847] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:53:14.278812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14d3ee-44c5-4ea6-9485-6f986d570746 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:14.278812 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14d3ee-44c5-4ea6-9485-6f986d570746 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:14.279187 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14d3ee-44c5-4ea6-9485-6f986d570746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:14.300569 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2f14d3ee-44c5-4ea6-9485-6f986d570746 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:53:14.301827 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2f14d3ee-44c5-4ea6-9485-6f986d570746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:14.309706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1426/2848] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:53:14.313950 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80277d64-96de-4b76-a9f4-8bb98fec4de4 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:14.324447 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80277d64-96de-4b76-a9f4-8bb98fec4de4 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:14.329452 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63b277-b0d2-48f2-9b73-c7b3738f4c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:14.330099 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63b277-b0d2-48f2-9b73-c7b3738f4c54 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:14.330381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63b277-b0d2-48f2-9b73-c7b3738f4c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:14.330783 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63b277-b0d2-48f2-9b73-c7b3738f4c54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:14.331491 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63b277-b0d2-48f2-9b73-c7b3738f4c54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:53:14.331800 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63b277-b0d2-48f2-9b73-c7b3738f4c54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:14.333454 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63b277-b0d2-48f2-9b73-c7b3738f4c54 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:14.345954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1423/2849] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:53:14.350371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14d3ee-44c5-4ea6-9485-6f986d570746 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:14.359771 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14d3ee-44c5-4ea6-9485-6f986d570746 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:14.364313 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28579be-88c4-4ffb-83e6-49fe8f3f24a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:14.364929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28579be-88c4-4ffb-83e6-49fe8f3f24a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:14.365207 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28579be-88c4-4ffb-83e6-49fe8f3f24a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:14.365501 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28579be-88c4-4ffb-83e6-49fe8f3f24a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:14.366179 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28579be-88c4-4ffb-83e6-49fe8f3f24a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:14.368534 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28579be-88c4-4ffb-83e6-49fe8f3f24a6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:14.397551 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1427/2850] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:53:14 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:53:14.402463 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63b277-b0d2-48f2-9b73-c7b3738f4c54 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:14.412656 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63b277-b0d2-48f2-9b73-c7b3738f4c54 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:14.457845 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:53:14.473718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9af72-3ea6-4e13-846c-624c69c20a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:14.474438 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9af72-3ea6-4e13-846c-624c69c20a1c admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:14.474740 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9af72-3ea6-4e13-846c-624c69c20a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:14.474998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9af72-3ea6-4e13-846c-624c69c20a1c admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:14.475805 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9af72-3ea6-4e13-846c-624c69c20a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:14.477769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9af72-3ea6-4e13-846c-624c69c20a1c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:14.501500 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1424/2851] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:53:14 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:53:14.504223 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28579be-88c4-4ffb-83e6-49fe8f3f24a6 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:14.515220 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28579be-88c4-4ffb-83e6-49fe8f3f24a6 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:14.519867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0b955-0bc5-45ce-872e-1908b7423fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:14.520483 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0b955-0bc5-45ce-872e-1908b7423fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:14.520830 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0b955-0bc5-45ce-872e-1908b7423fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:14.521188 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0b955-0bc5-45ce-872e-1908b7423fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:14.521972 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0b955-0bc5-45ce-872e-1908b7423fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:14.523601 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0b955-0bc5-45ce-872e-1908b7423fb3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:14.537075 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1428/2852] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:53:14.538434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9af72-3ea6-4e13-846c-624c69c20a1c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:14.549139 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9af72-3ea6-4e13-846c-624c69c20a1c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:14.553883 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4493865-4049-4fdc-aab5-5ebff730888a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:14.554524 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4493865-4049-4fdc-aab5-5ebff730888a admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:14.554802 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4493865-4049-4fdc-aab5-5ebff730888a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:14.555613 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4493865-4049-4fdc-aab5-5ebff730888a admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:14.556306 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4493865-4049-4fdc-aab5-5ebff730888a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:14.557903 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4493865-4049-4fdc-aab5-5ebff730888a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:14.569832 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1425/2853] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:53:14.576503 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0b955-0bc5-45ce-872e-1908b7423fb3 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:14.586748 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0b955-0bc5-45ce-872e-1908b7423fb3 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:14.591320 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6714a07-9906-4c87-88d0-4463c34f272b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:14.591982 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6714a07-9906-4c87-88d0-4463c34f272b admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:14.592265 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6714a07-9906-4c87-88d0-4463c34f272b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:14.592538 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6714a07-9906-4c87-88d0-4463c34f272b admin admin] PATH_INFO: `/v3/projects/77ba7a3ad9da4730990be14ba52f3982/users/0d51939ee985498f877578410774ed6b/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:14.608478 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6714a07-9906-4c87-88d0-4463c34f272b admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ba7a3ad9da4730990be14ba52f3982, user_id=0d51939ee985498f877578410774ed6b, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:14.611192 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6714a07-9906-4c87-88d0-4463c34f272b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:14.625063 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:53:14.639979 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1429/2854] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:53:14 2022] PUT /identity/v3/projects/77ba7a3ad9da4730990be14ba52f3982/users/0d51939ee985498f877578410774ed6b/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:53:14.665383 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab0049-3bb4-4f1d-914f-a2c3ea4300b1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:14.665709 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab0049-3bb4-4f1d-914f-a2c3ea4300b1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:14.665985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab0049-3bb4-4f1d-914f-a2c3ea4300b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:14.719014 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4fab0049-3bb4-4f1d-914f-a2c3ea4300b1 None None] MFA Rules not processed for user `0d51939ee985498f877578410774ed6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:53:14.720735 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4fab0049-3bb4-4f1d-914f-a2c3ea4300b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:14.815542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1426/2855] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:53:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:53:14.830456 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6714a07-9906-4c87-88d0-4463c34f272b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:14.832222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e6714a07-9906-4c87-88d0-4463c34f272b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:14.902144 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6714a07-9906-4c87-88d0-4463c34f272b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:14.906146 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f73285-081a-4a84-8376-f93d9aae4eab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:14.906770 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f73285-081a-4a84-8376-f93d9aae4eab service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:14.907109 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f73285-081a-4a84-8376-f93d9aae4eab service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:14.907384 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f73285-081a-4a84-8376-f93d9aae4eab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:14.918334 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f73285-081a-4a84-8376-f93d9aae4eab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:14.919806 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f73285-081a-4a84-8376-f93d9aae4eab service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:14.929245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1430/2856] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:53:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:53:14.954885 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab0049-3bb4-4f1d-914f-a2c3ea4300b1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:15.020876 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab0049-3bb4-4f1d-914f-a2c3ea4300b1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:15.025403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172385b-bff1-4ce2-993d-413217df9ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:15.026089 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172385b-bff1-4ce2-993d-413217df9ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:15.026364 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172385b-bff1-4ce2-993d-413217df9ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:15.026634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172385b-bff1-4ce2-993d-413217df9ac2 admin admin] PATH_INFO: `/v3/users/0d51939ee985498f877578410774ed6b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:15.027830 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6172385b-bff1-4ce2-993d-413217df9ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d51939ee985498f877578410774ed6b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:15.029763 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6172385b-bff1-4ce2-993d-413217df9ac2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:15.080726 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6172385b-bff1-4ce2-993d-413217df9ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d51939ee985498f877578410774ed6b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:15.089769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:53:15.096864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6172385b-bff1-4ce2-993d-413217df9ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d51939ee985498f877578410774ed6b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:15.098191 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6172385b-bff1-4ce2-993d-413217df9ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d51939ee985498f877578410774ed6b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:15.106982 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6172385b-bff1-4ce2-993d-413217df9ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d51939ee985498f877578410774ed6b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:15.114152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6172385b-bff1-4ce2-993d-413217df9ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d51939ee985498f877578410774ed6b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:15.121118 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1427/2857] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:53:14 2022] DELETE /identity/v3/users/0d51939ee985498f877578410774ed6b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:53:15.125914 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6106c-9af6-4ab7-98a4-4995716d8e78 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:15.126371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6106c-9af6-4ab7-98a4-4995716d8e78 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:15.126608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6106c-9af6-4ab7-98a4-4995716d8e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:15.145676 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-67f6106c-9af6-4ab7-98a4-4995716d8e78 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:53:15.146939 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-67f6106c-9af6-4ab7-98a4-4995716d8e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:15.215714 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1431/2858] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:53:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:53:15.225497 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172385b-bff1-4ce2-993d-413217df9ac2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:15.227626 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6172385b-bff1-4ce2-993d-413217df9ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:15.293384 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172385b-bff1-4ce2-993d-413217df9ac2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:15.297244 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee70159-5b59-47b4-97fa-02ec70ad28a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:15.297935 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee70159-5b59-47b4-97fa-02ec70ad28a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:15.298214 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee70159-5b59-47b4-97fa-02ec70ad28a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:15.298485 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee70159-5b59-47b4-97fa-02ec70ad28a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:15.309847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee70159-5b59-47b4-97fa-02ec70ad28a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:15.311291 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee70159-5b59-47b4-97fa-02ec70ad28a8 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:15.322565 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1428/2859] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:53:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:53:15.792447 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6106c-9af6-4ab7-98a4-4995716d8e78 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:15.794023 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-67f6106c-9af6-4ab7-98a4-4995716d8e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:15.806344 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6106c-9af6-4ab7-98a4-4995716d8e78 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:15.815635 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369cab6-76c4-49ff-b801-070c64a3fedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:15.815635 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369cab6-76c4-49ff-b801-070c64a3fedb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:15.815635 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369cab6-76c4-49ff-b801-070c64a3fedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:15.815635 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369cab6-76c4-49ff-b801-070c64a3fedb admin admin] PATH_INFO: `/v3/projects/77ba7a3ad9da4730990be14ba52f3982` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:15.828678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369cab6-76c4-49ff-b801-070c64a3fedb admin admin] RBAC: Authorizing `identity:delete_project(project_id=77ba7a3ad9da4730990be14ba52f3982)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:15.830789 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369cab6-76c4-49ff-b801-070c64a3fedb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:15.882647 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f369cab6-76c4-49ff-b801-070c64a3fedb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77ba7a3ad9da4730990be14ba52f3982', 'request_id': 'req-f369cab6-76c4-49ff-b801-070c64a3fedb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:15.890033 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f369cab6-76c4-49ff-b801-070c64a3fedb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77ba7a3ad9da4730990be14ba52f3982', 'request_id': 'req-f369cab6-76c4-49ff-b801-070c64a3fedb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:15.897929 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f369cab6-76c4-49ff-b801-070c64a3fedb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77ba7a3ad9da4730990be14ba52f3982', 'request_id': 'req-f369cab6-76c4-49ff-b801-070c64a3fedb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:15.913495 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f369cab6-76c4-49ff-b801-070c64a3fedb admin admin] The token cache is being invalidate because project 77ba7a3ad9da4730990be14ba52f3982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:53:15.913811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f369cab6-76c4-49ff-b801-070c64a3fedb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:15.914573 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f369cab6-76c4-49ff-b801-070c64a3fedb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:15.916112 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1432/2860] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:53:15 2022] DELETE /identity/v3/projects/77ba7a3ad9da4730990be14ba52f3982 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:53:20.122888 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee70159-5b59-47b4-97fa-02ec70ad28a8 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:20.125228 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-aee70159-5b59-47b4-97fa-02ec70ad28a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:20.138769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee70159-5b59-47b4-97fa-02ec70ad28a8 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:20.199767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a8e18-3017-46b0-99ec-1ee257adb513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:20.200405 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a8e18-3017-46b0-99ec-1ee257adb513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:20.200683 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a8e18-3017-46b0-99ec-1ee257adb513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:20.200954 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a8e18-3017-46b0-99ec-1ee257adb513 admin admin] PATH_INFO: `/v3/users/538d552b04a8446094d2e84952804795` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:20.202138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a8e18-3017-46b0-99ec-1ee257adb513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=538d552b04a8446094d2e84952804795)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:20.204130 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a8e18-3017-46b0-99ec-1ee257adb513 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:20.251424 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-be2a8e18-3017-46b0-99ec-1ee257adb513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '538d552b04a8446094d2e84952804795'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:20.262925 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-be2a8e18-3017-46b0-99ec-1ee257adb513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '538d552b04a8446094d2e84952804795'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:20.263603 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-be2a8e18-3017-46b0-99ec-1ee257adb513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '538d552b04a8446094d2e84952804795'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:20.270876 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-be2a8e18-3017-46b0-99ec-1ee257adb513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '538d552b04a8446094d2e84952804795'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:20.276255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:53:20.278430 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-be2a8e18-3017-46b0-99ec-1ee257adb513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '538d552b04a8446094d2e84952804795'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:20.286008 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1429/2861] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:53:20 2022] DELETE /identity/v3/users/538d552b04a8446094d2e84952804795 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:53:20.477658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369cab6-76c4-49ff-b801-070c64a3fedb admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:20.479103 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f369cab6-76c4-49ff-b801-070c64a3fedb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:20.493217 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369cab6-76c4-49ff-b801-070c64a3fedb admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:20.518632 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:53:20.553560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ff147-cf20-42c2-9cd2-e196d47343d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:20.554208 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ff147-cf20-42c2-9cd2-e196d47343d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:20.554492 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ff147-cf20-42c2-9cd2-e196d47343d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:20.554768 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ff147-cf20-42c2-9cd2-e196d47343d8 admin admin] PATH_INFO: `/v3/projects/ac7eca525bbe4443a7b99e0aa921050e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:20.555950 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ff147-cf20-42c2-9cd2-e196d47343d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac7eca525bbe4443a7b99e0aa921050e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:20.557900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ff147-cf20-42c2-9cd2-e196d47343d8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:20.603544 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-878ff147-cf20-42c2-9cd2-e196d47343d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac7eca525bbe4443a7b99e0aa921050e', 'request_id': 'req-878ff147-cf20-42c2-9cd2-e196d47343d8'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:20.613662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-878ff147-cf20-42c2-9cd2-e196d47343d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac7eca525bbe4443a7b99e0aa921050e', 'request_id': 'req-878ff147-cf20-42c2-9cd2-e196d47343d8'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:20.622535 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-878ff147-cf20-42c2-9cd2-e196d47343d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac7eca525bbe4443a7b99e0aa921050e', 'request_id': 'req-878ff147-cf20-42c2-9cd2-e196d47343d8'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:20.632959 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-878ff147-cf20-42c2-9cd2-e196d47343d8 admin admin] The token cache is being invalidate because project ac7eca525bbe4443a7b99e0aa921050e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:53:20.633295 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-878ff147-cf20-42c2-9cd2-e196d47343d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:20.633964 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-878ff147-cf20-42c2-9cd2-e196d47343d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:20.635488 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1433/2862] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:53:20 2022] DELETE /identity/v3/projects/ac7eca525bbe4443a7b99e0aa921050e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:53:20.639965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a8e18-3017-46b0-99ec-1ee257adb513 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:20.641434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-be2a8e18-3017-46b0-99ec-1ee257adb513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:20.655244 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a8e18-3017-46b0-99ec-1ee257adb513 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:20.718480 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ff862-81b4-48ad-8077-4a76ea6689f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:20.719155 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ff862-81b4-48ad-8077-4a76ea6689f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:20.719434 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ff862-81b4-48ad-8077-4a76ea6689f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:20.719707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ff862-81b4-48ad-8077-4a76ea6689f3 admin admin] PATH_INFO: `/v3/users/14be2a82b6e94e3b8d815bfed390faa3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:20.720864 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ff862-81b4-48ad-8077-4a76ea6689f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14be2a82b6e94e3b8d815bfed390faa3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:20.722842 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ff862-81b4-48ad-8077-4a76ea6689f3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:20.779361 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-609ff862-81b4-48ad-8077-4a76ea6689f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14be2a82b6e94e3b8d815bfed390faa3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:20.792652 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-609ff862-81b4-48ad-8077-4a76ea6689f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14be2a82b6e94e3b8d815bfed390faa3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:20.793653 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-609ff862-81b4-48ad-8077-4a76ea6689f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14be2a82b6e94e3b8d815bfed390faa3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:20.808993 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-609ff862-81b4-48ad-8077-4a76ea6689f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14be2a82b6e94e3b8d815bfed390faa3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:20.817239 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-609ff862-81b4-48ad-8077-4a76ea6689f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14be2a82b6e94e3b8d815bfed390faa3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:20.824832 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1430/2863] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 10:53:20 2022] DELETE /identity/v3/users/14be2a82b6e94e3b8d815bfed390faa3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:53:21.021228 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ff147-cf20-42c2-9cd2-e196d47343d8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:21.022734 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-878ff147-cf20-42c2-9cd2-e196d47343d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:21.036241 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ff147-cf20-42c2-9cd2-e196d47343d8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:21.098807 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d47767-e624-4c63-aea5-ced1d426d1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:21.099472 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d47767-e624-4c63-aea5-ced1d426d1bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:21.099747 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d47767-e624-4c63-aea5-ced1d426d1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:21.100037 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d47767-e624-4c63-aea5-ced1d426d1bb admin admin] PATH_INFO: `/v3/projects/30136a0a94d2491f8621ad2f56caa5a7` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:21.101215 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d47767-e624-4c63-aea5-ced1d426d1bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=30136a0a94d2491f8621ad2f56caa5a7)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:21.103782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d47767-e624-4c63-aea5-ced1d426d1bb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:21.154240 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-65d47767-e624-4c63-aea5-ced1d426d1bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30136a0a94d2491f8621ad2f56caa5a7', 'request_id': 'req-65d47767-e624-4c63-aea5-ced1d426d1bb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:21.161270 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-65d47767-e624-4c63-aea5-ced1d426d1bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30136a0a94d2491f8621ad2f56caa5a7', 'request_id': 'req-65d47767-e624-4c63-aea5-ced1d426d1bb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:21.165897 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:53:21.171809 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-65d47767-e624-4c63-aea5-ced1d426d1bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30136a0a94d2491f8621ad2f56caa5a7', 'request_id': 'req-65d47767-e624-4c63-aea5-ced1d426d1bb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:21.184665 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-65d47767-e624-4c63-aea5-ced1d426d1bb admin admin] The token cache is being invalidate because project 30136a0a94d2491f8621ad2f56caa5a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 10:53:21.184954 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-65d47767-e624-4c63-aea5-ced1d426d1bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:21.185651 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-65d47767-e624-4c63-aea5-ced1d426d1bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 10:53:21.187222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1434/2864] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 10:53:21 2022] DELETE /identity/v3/projects/30136a0a94d2491f8621ad2f56caa5a7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:53:21.192998 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02094fc0-42ed-44b8-b64f-358c325331ce None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:21.193553 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02094fc0-42ed-44b8-b64f-358c325331ce None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:21.193796 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02094fc0-42ed-44b8-b64f-358c325331ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:21.214135 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-02094fc0-42ed-44b8-b64f-358c325331ce None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:53:21.215360 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-02094fc0-42ed-44b8-b64f-358c325331ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:21.282035 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1431/2865] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:53:21.290706 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c81043-293d-4499-92a2-1e00ae13c86d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:21.290986 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c81043-293d-4499-92a2-1e00ae13c86d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:21.291271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c81043-293d-4499-92a2-1e00ae13c86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:21.312190 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a8c81043-293d-4499-92a2-1e00ae13c86d None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:53:21.313458 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a8c81043-293d-4499-92a2-1e00ae13c86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:21.321476 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1435/2866] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:53:21.325850 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02094fc0-42ed-44b8-b64f-358c325331ce None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:21.335666 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02094fc0-42ed-44b8-b64f-358c325331ce None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:21.340769 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb3b01-81ca-4532-859e-ccf45a800680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:21.341399 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb3b01-81ca-4532-859e-ccf45a800680 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:21.341706 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb3b01-81ca-4532-859e-ccf45a800680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:21.341978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb3b01-81ca-4532-859e-ccf45a800680 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:21.342707 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb3b01-81ca-4532-859e-ccf45a800680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 10:53:21.343013 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb3b01-81ca-4532-859e-ccf45a800680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:21.344628 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb3b01-81ca-4532-859e-ccf45a800680 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:21.357150 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1432/2867] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 3 10:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 10:53:21.361597 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c81043-293d-4499-92a2-1e00ae13c86d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:21.371466 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c81043-293d-4499-92a2-1e00ae13c86d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:21.376343 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8d0a8-1ae0-44de-a980-bbeb44d62cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:21.376967 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8d0a8-1ae0-44de-a980-bbeb44d62cee admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:21.377245 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8d0a8-1ae0-44de-a980-bbeb44d62cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:21.377547 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8d0a8-1ae0-44de-a980-bbeb44d62cee admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:21.378208 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8d0a8-1ae0-44de-a980-bbeb44d62cee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:21.380093 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8d0a8-1ae0-44de-a980-bbeb44d62cee admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:21.407811 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1436/2868] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:53:21 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:53:21.411914 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb3b01-81ca-4532-859e-ccf45a800680 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:21.417756 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:53:21.424051 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb3b01-81ca-4532-859e-ccf45a800680 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:21.482486 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78df83e-7eb6-4330-bc70-5cbeda80503d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:21.483109 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78df83e-7eb6-4330-bc70-5cbeda80503d admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:21.483390 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78df83e-7eb6-4330-bc70-5cbeda80503d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:21.483661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78df83e-7eb6-4330-bc70-5cbeda80503d admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:21.484320 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78df83e-7eb6-4330-bc70-5cbeda80503d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:21.486258 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78df83e-7eb6-4330-bc70-5cbeda80503d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:21.509957 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1433/2869] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:53:21 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:53:21.514345 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8d0a8-1ae0-44de-a980-bbeb44d62cee admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:21.525378 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8d0a8-1ae0-44de-a980-bbeb44d62cee admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:21.530021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50430f27-d906-4e42-8b96-cb3929eda30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:21.530633 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50430f27-d906-4e42-8b96-cb3929eda30e admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:21.530913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50430f27-d906-4e42-8b96-cb3929eda30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:21.531181 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50430f27-d906-4e42-8b96-cb3929eda30e admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:21.531867 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50430f27-d906-4e42-8b96-cb3929eda30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:21.533453 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50430f27-d906-4e42-8b96-cb3929eda30e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:21.544658 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1437/2870] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:53:21.548846 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78df83e-7eb6-4330-bc70-5cbeda80503d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:21.559135 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78df83e-7eb6-4330-bc70-5cbeda80503d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:21.563913 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ef96b-dc7b-4877-976e-f9b6b96ac6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:21.564532 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ef96b-dc7b-4877-976e-f9b6b96ac6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:21.564812 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ef96b-dc7b-4877-976e-f9b6b96ac6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:21.565086 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ef96b-dc7b-4877-976e-f9b6b96ac6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:21.565946 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ef96b-dc7b-4877-976e-f9b6b96ac6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:21.567478 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ef96b-dc7b-4877-976e-f9b6b96ac6d0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:21.579028 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1434/2871] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:53:21.583961 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50430f27-d906-4e42-8b96-cb3929eda30e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:21.594940 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50430f27-d906-4e42-8b96-cb3929eda30e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:21.599435 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffd6aa-b76d-492b-ad90-8e309644923d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:21.600087 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffd6aa-b76d-492b-ad90-8e309644923d admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:21.600366 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffd6aa-b76d-492b-ad90-8e309644923d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:21.600640 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffd6aa-b76d-492b-ad90-8e309644923d admin admin] PATH_INFO: `/v3/projects/ad63d5a5bdb44dd6af0c25d524f38508/users/7445b3f8940244c2abe1e3f1573bcd8a/roles/82be5cfedcb3407f8d14522ef87247ac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:21.614955 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffd6aa-b76d-492b-ad90-8e309644923d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad63d5a5bdb44dd6af0c25d524f38508, user_id=7445b3f8940244c2abe1e3f1573bcd8a, role_id=82be5cfedcb3407f8d14522ef87247ac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:21.617871 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffd6aa-b76d-492b-ad90-8e309644923d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:21.630269 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1438/2872] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:53:21 2022] PUT /identity/v3/projects/ad63d5a5bdb44dd6af0c25d524f38508/users/7445b3f8940244c2abe1e3f1573bcd8a/roles/82be5cfedcb3407f8d14522ef87247ac => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:53:21.635833 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f1306-8817-4021-8d75-23450e0b5087 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:21.636146 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f1306-8817-4021-8d75-23450e0b5087 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:21.636470 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f1306-8817-4021-8d75-23450e0b5087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:21.656865 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5e0f1306-8817-4021-8d75-23450e0b5087 None None] MFA Rules not processed for user `fb1a77890b95445982cf98d4ad77cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:53:21.658164 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5e0f1306-8817-4021-8d75-23450e0b5087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:21.718296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1435/2873] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:53:21.727909 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffd6aa-b76d-492b-ad90-8e309644923d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:21.729657 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7dffd6aa-b76d-492b-ad90-8e309644923d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:21.784848 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffd6aa-b76d-492b-ad90-8e309644923d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:21.788486 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990d9ae-bab7-4c47-8c13-a26615f14926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:21.789106 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990d9ae-bab7-4c47-8c13-a26615f14926 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:21.789390 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990d9ae-bab7-4c47-8c13-a26615f14926 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:21.789821 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990d9ae-bab7-4c47-8c13-a26615f14926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:21.798800 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990d9ae-bab7-4c47-8c13-a26615f14926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:21.800219 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990d9ae-bab7-4c47-8c13-a26615f14926 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:21.810225 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1439/2874] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:53:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:53:25.733625 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b379224-b789-4b46-bd0d-ae0840714336 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:25.734175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b379224-b789-4b46-bd0d-ae0840714336 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:25.734175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b379224-b789-4b46-bd0d-ae0840714336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:25.767844 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2b379224-b789-4b46-bd0d-ae0840714336 None None] MFA Rules not processed for user `7445b3f8940244c2abe1e3f1573bcd8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:53:25.769389 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2b379224-b789-4b46-bd0d-ae0840714336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:25.838946 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1436/2875] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:53:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:53:25.843584 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990d9ae-bab7-4c47-8c13-a26615f14926 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:25.853936 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990d9ae-bab7-4c47-8c13-a26615f14926 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:25.858712 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fedeae-970d-4076-b7fb-7bb2dbd9c22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:25.859427 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fedeae-970d-4076-b7fb-7bb2dbd9c22f admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:25.859728 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fedeae-970d-4076-b7fb-7bb2dbd9c22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:25.859984 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fedeae-970d-4076-b7fb-7bb2dbd9c22f admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:25.860695 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fedeae-970d-4076-b7fb-7bb2dbd9c22f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:25.862623 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fedeae-970d-4076-b7fb-7bb2dbd9c22f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:25.891726 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1440/2876] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 3 10:53:25 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 10:53:25.895854 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b379224-b789-4b46-bd0d-ae0840714336 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:25.906991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b379224-b789-4b46-bd0d-ae0840714336 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:25.916610 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:53:25.968403 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886c975-21e7-4a70-81c7-6058fe4a161a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:25.969041 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886c975-21e7-4a70-81c7-6058fe4a161a admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:25.969349 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886c975-21e7-4a70-81c7-6058fe4a161a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:25.969664 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886c975-21e7-4a70-81c7-6058fe4a161a admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:25.970348 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886c975-21e7-4a70-81c7-6058fe4a161a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:25.972288 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886c975-21e7-4a70-81c7-6058fe4a161a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:25.996103 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1437/2877] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 3 10:53:25 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 10:53:25.999923 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fedeae-970d-4076-b7fb-7bb2dbd9c22f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:26.012020 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fedeae-970d-4076-b7fb-7bb2dbd9c22f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:26.016609 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c167b-6b16-4e2f-89aa-d7ea814c5022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:26.017225 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c167b-6b16-4e2f-89aa-d7ea814c5022 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:26.017544 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c167b-6b16-4e2f-89aa-d7ea814c5022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:26.017823 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c167b-6b16-4e2f-89aa-d7ea814c5022 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:26.018513 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c167b-6b16-4e2f-89aa-d7ea814c5022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:26.020108 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c167b-6b16-4e2f-89aa-d7ea814c5022 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:26.031323 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1441/2878] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:53:26.035682 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886c975-21e7-4a70-81c7-6058fe4a161a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:26.046117 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886c975-21e7-4a70-81c7-6058fe4a161a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:26.050905 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab83caa-dc37-44c9-9b94-27e3a92ac876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:26.051594 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab83caa-dc37-44c9-9b94-27e3a92ac876 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:26.051876 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab83caa-dc37-44c9-9b94-27e3a92ac876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:26.052175 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab83caa-dc37-44c9-9b94-27e3a92ac876 admin admin] PATH_INFO: `/v3/projects/93903bcbf1334650bdc816450f2ecc01/users/97a484f269514769a901b18e79fd54e7/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:26.068612 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab83caa-dc37-44c9-9b94-27e3a92ac876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93903bcbf1334650bdc816450f2ecc01, user_id=97a484f269514769a901b18e79fd54e7, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:26.071931 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab83caa-dc37-44c9-9b94-27e3a92ac876 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:26.085924 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1438/2879] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:53:26 2022] PUT /identity/v3/projects/93903bcbf1334650bdc816450f2ecc01/users/97a484f269514769a901b18e79fd54e7/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:53:26.090070 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c167b-6b16-4e2f-89aa-d7ea814c5022 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:26.100538 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c167b-6b16-4e2f-89aa-d7ea814c5022 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:26.148817 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:53:26.167869 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4638e30-4dd0-4b98-b700-d1a96c0e446c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:26.168500 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4638e30-4dd0-4b98-b700-d1a96c0e446c admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:26.168779 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4638e30-4dd0-4b98-b700-d1a96c0e446c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:26.169050 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4638e30-4dd0-4b98-b700-d1a96c0e446c admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:26.169782 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4638e30-4dd0-4b98-b700-d1a96c0e446c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:26.171371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4638e30-4dd0-4b98-b700-d1a96c0e446c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:26.182339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1442/2880] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 3 10:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 10:53:26.186794 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab83caa-dc37-44c9-9b94-27e3a92ac876 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:26.197437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab83caa-dc37-44c9-9b94-27e3a92ac876 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:26.204233 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937aa68b-a381-4931-9622-22ba59206a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:26.204992 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937aa68b-a381-4931-9622-22ba59206a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:26.205280 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937aa68b-a381-4931-9622-22ba59206a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:26.205617 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937aa68b-a381-4931-9622-22ba59206a7e admin admin] PATH_INFO: `/v3/projects/93903bcbf1334650bdc816450f2ecc01/users/97a484f269514769a901b18e79fd54e7/roles/01714797cab143e1a7a150c068fa9315` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:26.207629 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937aa68b-a381-4931-9622-22ba59206a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=93903bcbf1334650bdc816450f2ecc01, user_id=97a484f269514769a901b18e79fd54e7, role_id=01714797cab143e1a7a150c068fa9315)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:26.210278 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937aa68b-a381-4931-9622-22ba59206a7e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:26.221165 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1439/2881] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 3 10:53:26 2022] PUT /identity/v3/projects/93903bcbf1334650bdc816450f2ecc01/users/97a484f269514769a901b18e79fd54e7/roles/01714797cab143e1a7a150c068fa9315 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 10:53:30.679022 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16772fab-f819-4e99-ada0-80d3e8f3dc31 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:30.679608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16772fab-f819-4e99-ada0-80d3e8f3dc31 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:30.679608 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16772fab-f819-4e99-ada0-80d3e8f3dc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:30.747352 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-16772fab-f819-4e99-ada0-80d3e8f3dc31 None None] MFA Rules not processed for user `97a484f269514769a901b18e79fd54e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:53:30.749556 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-16772fab-f819-4e99-ada0-80d3e8f3dc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:30.858431 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937aa68b-a381-4931-9622-22ba59206a7e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:30.858857 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1443/2882] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:53:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:53:30.859857 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-937aa68b-a381-4931-9622-22ba59206a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:30.980583 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937aa68b-a381-4931-9622-22ba59206a7e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:30.986037 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142e3ef-23ff-496b-b16b-91581eac48ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:30.986661 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142e3ef-23ff-496b-b16b-91581eac48ab service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:30.986941 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142e3ef-23ff-496b-b16b-91581eac48ab service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:30.987211 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142e3ef-23ff-496b-b16b-91581eac48ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:31.047851 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142e3ef-23ff-496b-b16b-91581eac48ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:31.049691 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142e3ef-23ff-496b-b16b-91581eac48ab service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:31.055586 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:53:31.063163 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1440/2883] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:53:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:53:40.839149 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=39 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:53:46.334254 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bbbe1-06e8-4108-a9b5-9326b5d79e7e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:46.334899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bbbe1-06e8-4108-a9b5-9326b5d79e7e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:46.334899 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bbbe1-06e8-4108-a9b5-9326b5d79e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:46.354222 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ac4bbbe1-06e8-4108-a9b5-9326b5d79e7e None None] MFA Rules not processed for user `7445b3f8940244c2abe1e3f1573bcd8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:53:46.355434 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ac4bbbe1-06e8-4108-a9b5-9326b5d79e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:46.363069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1444/2884] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:53:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:53:46.405704 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142e3ef-23ff-496b-b16b-91581eac48ab service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:46.407126 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1142e3ef-23ff-496b-b16b-91581eac48ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:46.463985 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142e3ef-23ff-496b-b16b-91581eac48ab service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:46.467647 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432b923-cc86-4a49-93b1-33c0d9f732a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b5f547a388c4f64aeb95a7435818741', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:46.468255 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432b923-cc86-4a49-93b1-33c0d9f732a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:46.468528 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432b923-cc86-4a49-93b1-33c0d9f732a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:46.468797 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432b923-cc86-4a49-93b1-33c0d9f732a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:46.470552 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8432b923-cc86-4a49-93b1-33c0d9f732a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:46.483767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432b923-cc86-4a49-93b1-33c0d9f732a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:46.485114 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432b923-cc86-4a49-93b1-33c0d9f732a8 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:46.497187 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1441/2885] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:53:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:53:47.230543 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bbbe1-06e8-4108-a9b5-9326b5d79e7e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:53:47.235076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ac4bbbe1-06e8-4108-a9b5-9326b5d79e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:47.306901 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bbbe1-06e8-4108-a9b5-9326b5d79e7e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:53:47.310873 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ef84b-ddf0-4fdb-aef5-14f37a3c70d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:53:47.311603 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ef84b-ddf0-4fdb-aef5-14f37a3c70d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:53:47.311954 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ef84b-ddf0-4fdb-aef5-14f37a3c70d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:53:47.312306 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ef84b-ddf0-4fdb-aef5-14f37a3c70d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:53:47.314218 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-874ef84b-ddf0-4fdb-aef5-14f37a3c70d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:53:47.328410 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ef84b-ddf0-4fdb-aef5-14f37a3c70d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:53:47.329880 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ef84b-ddf0-4fdb-aef5-14f37a3c70d9 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:53:47.341294 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1445/2886] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:53:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:53:56.491738 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=26 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:53:57.336146 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=29 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:54:01.972144 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432b923-cc86-4a49-93b1-33c0d9f732a8 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:54:01.982138 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432b923-cc86-4a49-93b1-33c0d9f732a8 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:54:01.985839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbc34b-9c3e-4afd-af7c-9b7f63e93470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:54:01.986480 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbc34b-9c3e-4afd-af7c-9b7f63e93470 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:54:01.986765 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbc34b-9c3e-4afd-af7c-9b7f63e93470 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:54:01.987048 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbc34b-9c3e-4afd-af7c-9b7f63e93470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:54:01.996965 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfbc34b-9c3e-4afd-af7c-9b7f63e93470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:54:01.998454 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfbc34b-9c3e-4afd-af7c-9b7f63e93470 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:54:02.007907 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1442/2887] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:54:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:54:12.003629 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:54:45.162176 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ef84b-ddf0-4fdb-aef5-14f37a3c70d9 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:54:45.171069 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ef84b-ddf0-4fdb-aef5-14f37a3c70d9 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:54:45.175389 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e888da6-7330-412e-8b13-b58e29f1b685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:54:45.176009 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e888da6-7330-412e-8b13-b58e29f1b685 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:54:45.176283 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e888da6-7330-412e-8b13-b58e29f1b685 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:54:45.176552 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e888da6-7330-412e-8b13-b58e29f1b685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:54:45.178339 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0e888da6-7330-412e-8b13-b58e29f1b685 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:54:45.188080 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e888da6-7330-412e-8b13-b58e29f1b685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:54:45.189489 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e888da6-7330-412e-8b13-b58e29f1b685 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:54:45.197387 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1446/2888] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:54:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:54:55.193858 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:56:02.353025 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbc34b-9c3e-4afd-af7c-9b7f63e93470 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:56:02.354983 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3cfbc34b-9c3e-4afd-af7c-9b7f63e93470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:56:02.405839 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbc34b-9c3e-4afd-af7c-9b7f63e93470 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:56:02.418558 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608707b-02df-45be-9698-3364366f1165 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:56:02.419224 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608707b-02df-45be-9698-3364366f1165 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:56:02.419489 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608707b-02df-45be-9698-3364366f1165 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:56:02.420790 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608707b-02df-45be-9698-3364366f1165 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:56:02.424519 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d608707b-02df-45be-9698-3364366f1165 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:56:02.444908 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608707b-02df-45be-9698-3364366f1165 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:56:02.446934 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608707b-02df-45be-9698-3364366f1165 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:56:02.461533 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1443/2889] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:56:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:56:03.585519 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b88b7d-d7c6-4eb4-8315-764f2ddf1752 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:56:03.586041 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b88b7d-d7c6-4eb4-8315-764f2ddf1752 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:56:03.586145 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b88b7d-d7c6-4eb4-8315-764f2ddf1752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:56:03.607098 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-68b88b7d-d7c6-4eb4-8315-764f2ddf1752 None None] MFA Rules not processed for user `7445b3f8940244c2abe1e3f1573bcd8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:56:03.608370 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-68b88b7d-d7c6-4eb4-8315-764f2ddf1752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:56:03.615593 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1447/2890] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:56:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 10:56:04.409406 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608707b-02df-45be-9698-3364366f1165 service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:56:04.411167 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d608707b-02df-45be-9698-3364366f1165 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:56:04.485718 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608707b-02df-45be-9698-3364366f1165 service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:56:04.491502 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a65af5-a96b-4782-9a53-4a17cb7402d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2c5536eddd346d692822a2f3befb48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:56:04.492145 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a65af5-a96b-4782-9a53-4a17cb7402d1 service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:56:04.492458 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a65af5-a96b-4782-9a53-4a17cb7402d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:56:04.492736 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a65af5-a96b-4782-9a53-4a17cb7402d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:56:04.494880 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d5a65af5-a96b-4782-9a53-4a17cb7402d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:56:04.507767 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a65af5-a96b-4782-9a53-4a17cb7402d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:56:04.509357 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a65af5-a96b-4782-9a53-4a17cb7402d1 service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:56:04.520241 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1444/2891] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:56:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:56:13.600420 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:56:14.515478 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=38 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:56:16.300366 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b88b7d-d7c6-4eb4-8315-764f2ddf1752 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:56:16.313862 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b88b7d-d7c6-4eb4-8315-764f2ddf1752 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:56:16.319452 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b0ae0-99c8-4808-bdb9-d0770d30890c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:56:16.320100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b0ae0-99c8-4808-bdb9-d0770d30890c service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:56:16.320385 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b0ae0-99c8-4808-bdb9-d0770d30890c service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:56:16.320678 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b0ae0-99c8-4808-bdb9-d0770d30890c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:56:16.322611 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-164b0ae0-99c8-4808-bdb9-d0770d30890c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:56:16.336248 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b0ae0-99c8-4808-bdb9-d0770d30890c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:56:16.337916 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b0ae0-99c8-4808-bdb9-d0770d30890c service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:56:16.352516 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1448/2892] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:56:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:56:26.345661 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:56:54.402322 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a65af5-a96b-4782-9a53-4a17cb7402d1 service placement] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:56:54.412967 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a65af5-a96b-4782-9a53-4a17cb7402d1 service placement] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:56:54.418033 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff804b9-c727-4e7b-b9d2-8b2f5d511f65 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2c5536eddd346d692822a2f3befb48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:56:54.418643 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff804b9-c727-4e7b-b9d2-8b2f5d511f65 service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:56:54.418927 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff804b9-c727-4e7b-b9d2-8b2f5d511f65 service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:56:54.419201 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff804b9-c727-4e7b-b9d2-8b2f5d511f65 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:56:54.420981 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bff804b9-c727-4e7b-b9d2-8b2f5d511f65 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:56:54.435821 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff804b9-c727-4e7b-b9d2-8b2f5d511f65 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:56:54.438120 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff804b9-c727-4e7b-b9d2-8b2f5d511f65 service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:56:54.449114 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1445/2893] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:56:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:57:04.444567 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:57:58.142704 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9f18d-ad21-47f1-9435-3ca72d31ef71 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:57:58.143238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9f18d-ad21-47f1-9435-3ca72d31ef71 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:57:58.143238 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9f18d-ad21-47f1-9435-3ca72d31ef71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:57:58.162086 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-98f9f18d-ad21-47f1-9435-3ca72d31ef71 None None] MFA Rules not processed for user `7445b3f8940244c2abe1e3f1573bcd8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 10:57:58.163271 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-98f9f18d-ad21-47f1-9435-3ca72d31ef71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:57:58.170612 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1449/2894] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 3 10:57:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 10:57:58.486618 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff804b9-c727-4e7b-b9d2-8b2f5d511f65 service placement] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:57:58.496500 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff804b9-c727-4e7b-b9d2-8b2f5d511f65 service placement] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:57:58.500162 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8d3ca-59e6-4b25-8cda-45cafff130a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:57:58.500791 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8d3ca-59e6-4b25-8cda-45cafff130a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:57:58.501068 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8d3ca-59e6-4b25-8cda-45cafff130a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:57:58.501338 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8d3ca-59e6-4b25-8cda-45cafff130a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:57:58.503085 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c1c8d3ca-59e6-4b25-8cda-45cafff130a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:57:58.512851 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8d3ca-59e6-4b25-8cda-45cafff130a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:57:58.514284 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8d3ca-59e6-4b25-8cda-45cafff130a6 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:57:58.522133 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1446/2895] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:57:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:58:08.157338 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:58:08.518572 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:58:17.360220 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9f18d-ad21-47f1-9435-3ca72d31ef71 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:58:17.373481 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9f18d-ad21-47f1-9435-3ca72d31ef71 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:58:17.379409 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491dcf1-e0a2-4a5a-be6f-2804232eb454 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e66f25cc329f418eba6bd637bf3e8a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:58:17.380109 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491dcf1-e0a2-4a5a-be6f-2804232eb454 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:58:17.380393 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491dcf1-e0a2-4a5a-be6f-2804232eb454 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:58:17.380664 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491dcf1-e0a2-4a5a-be6f-2804232eb454 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:58:17.382492 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f491dcf1-e0a2-4a5a-be6f-2804232eb454 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 10:58:17.407411 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491dcf1-e0a2-4a5a-be6f-2804232eb454 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:58:17.408662 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491dcf1-e0a2-4a5a-be6f-2804232eb454 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:58:17.422396 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1450/2896] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:58:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:58:27.417100 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:59:44.318179 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8d3ca-59e6-4b25-8cda-45cafff130a6 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:59:44.326872 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8d3ca-59e6-4b25-8cda-45cafff130a6 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:59:44.331118 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03939128-dd6e-4713-8245-8b017adbc1ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:59:44.331722 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03939128-dd6e-4713-8245-8b017adbc1ae service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:59:44.332002 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03939128-dd6e-4713-8245-8b017adbc1ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:59:44.332275 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03939128-dd6e-4713-8245-8b017adbc1ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:59:44.340363 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03939128-dd6e-4713-8245-8b017adbc1ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:59:44.341804 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03939128-dd6e-4713-8245-8b017adbc1ae service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:59:44.350462 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1447/2897] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:59:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:59:45.274317 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491dcf1-e0a2-4a5a-be6f-2804232eb454 service nova] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:59:45.285230 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491dcf1-e0a2-4a5a-be6f-2804232eb454 service nova] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:59:45.289875 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655cc27-2e1c-4f63-a906-278aa91c0879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:59:45.290864 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655cc27-2e1c-4f63-a906-278aa91c0879 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:59:45.291219 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655cc27-2e1c-4f63-a906-278aa91c0879 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:59:45.291568 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655cc27-2e1c-4f63-a906-278aa91c0879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:59:45.301634 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655cc27-2e1c-4f63-a906-278aa91c0879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:59:45.303686 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655cc27-2e1c-4f63-a906-278aa91c0879 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:59:45.314346 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1451/2898] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:59:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:59:45.415647 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03939128-dd6e-4713-8245-8b017adbc1ae service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:59:45.427012 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03939128-dd6e-4713-8245-8b017adbc1ae service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:59:45.433101 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a329b5-d2c6-4d2c-95b4-7ed9afb69631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:59:45.433826 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a329b5-d2c6-4d2c-95b4-7ed9afb69631 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:59:45.434232 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a329b5-d2c6-4d2c-95b4-7ed9afb69631 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:59:45.434694 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a329b5-d2c6-4d2c-95b4-7ed9afb69631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:59:45.448875 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a329b5-d2c6-4d2c-95b4-7ed9afb69631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:59:45.450615 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a329b5-d2c6-4d2c-95b4-7ed9afb69631 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:59:45.465033 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1448/2899] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:59:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 10:59:46.652530 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655cc27-2e1c-4f63-a906-278aa91c0879 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:59:46.662903 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655cc27-2e1c-4f63-a906-278aa91c0879 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:59:46.666618 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea896aef-d056-4dc5-902c-57c4b9bb88a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:59:46.667234 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea896aef-d056-4dc5-902c-57c4b9bb88a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:59:46.667514 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea896aef-d056-4dc5-902c-57c4b9bb88a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:59:46.667785 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea896aef-d056-4dc5-902c-57c4b9bb88a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:59:46.679250 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea896aef-d056-4dc5-902c-57c4b9bb88a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:59:46.680883 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea896aef-d056-4dc5-902c-57c4b9bb88a3 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:59:46.693904 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1452/2900] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:59:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 10:59:55.458692 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=12 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:59:56.687605 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=12 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 10:59:57.624190 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a329b5-d2c6-4d2c-95b4-7ed9afb69631 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 10:59:57.635385 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a329b5-d2c6-4d2c-95b4-7ed9afb69631 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 10:59:57.639132 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882513b-771d-4fb4-8b52-5a5fb5b74e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c211cc19a2045bd9383d31c7e81ceac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 10:59:57.639746 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882513b-771d-4fb4-8b52-5a5fb5b74e49 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 10:59:57.640023 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882513b-771d-4fb4-8b52-5a5fb5b74e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 10:59:57.640292 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882513b-771d-4fb4-8b52-5a5fb5b74e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 10:59:57.703079 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882513b-771d-4fb4-8b52-5a5fb5b74e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 10:59:57.705218 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882513b-771d-4fb4-8b52-5a5fb5b74e49 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 10:59:57.715088 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1449/2901] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 10:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 11:00:00.882913 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea896aef-d056-4dc5-902c-57c4b9bb88a3 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 11:00:00.902231 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea896aef-d056-4dc5-902c-57c4b9bb88a3 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 11:00:00.911672 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1bd76-f009-40f0-8d44-dced3491fa11 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2c5536eddd346d692822a2f3befb48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 11:00:00.912613 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1bd76-f009-40f0-8d44-dced3491fa11 service placement] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 11:00:00.912907 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1bd76-f009-40f0-8d44-dced3491fa11 service placement] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 11:00:00.913182 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1bd76-f009-40f0-8d44-dced3491fa11 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 11:00:00.915423 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e3d1bd76-f009-40f0-8d44-dced3491fa11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 11:00:00.935883 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1bd76-f009-40f0-8d44-dced3491fa11 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 11:00:00.937645 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1bd76-f009-40f0-8d44-dced3491fa11 service placement] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 11:00:00.953360 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1453/2902] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 11:00:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 11:00:04.191483 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882513b-771d-4fb4-8b52-5a5fb5b74e49 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 11:00:04.212064 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882513b-771d-4fb4-8b52-5a5fb5b74e49 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 11:00:04.218750 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce48687-9cdb-48c0-8118-93b8c5d8d5b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2c5536eddd346d692822a2f3befb48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 11:00:04.220923 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce48687-9cdb-48c0-8118-93b8c5d8d5b0 service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 11:00:04.221206 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce48687-9cdb-48c0-8118-93b8c5d8d5b0 service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 11:00:04.221542 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce48687-9cdb-48c0-8118-93b8c5d8d5b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 11:00:04.223597 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0ce48687-9cdb-48c0-8118-93b8c5d8d5b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 11:00:04.239381 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce48687-9cdb-48c0-8118-93b8c5d8d5b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 11:00:04.240837 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce48687-9cdb-48c0-8118-93b8c5d8d5b0 service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 11:00:04.252159 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1bd76-f009-40f0-8d44-dced3491fa11 service placement] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 11:00:04.261196 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1450/2903] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 11:00:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 11:00:04.274893 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1bd76-f009-40f0-8d44-dced3491fa11 service placement] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 11:00:04.283557 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73eb607-61b2-4576-a956-0dd6bfdf1bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 11:00:04.284454 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73eb607-61b2-4576-a956-0dd6bfdf1bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 11:00:04.284740 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73eb607-61b2-4576-a956-0dd6bfdf1bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 11:00:04.285021 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73eb607-61b2-4576-a956-0dd6bfdf1bc8 admin admin] PATH_INFO: `/v3/users/7445b3f8940244c2abe1e3f1573bcd8a` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 11:00:04.286417 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73eb607-61b2-4576-a956-0dd6bfdf1bc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7445b3f8940244c2abe1e3f1573bcd8a)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 11:00:04.288432 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73eb607-61b2-4576-a956-0dd6bfdf1bc8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 11:00:04.345561 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e73eb607-61b2-4576-a956-0dd6bfdf1bc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7445b3f8940244c2abe1e3f1573bcd8a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:04.352560 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e73eb607-61b2-4576-a956-0dd6bfdf1bc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7445b3f8940244c2abe1e3f1573bcd8a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:04.360598 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e73eb607-61b2-4576-a956-0dd6bfdf1bc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7445b3f8940244c2abe1e3f1573bcd8a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:04.361214 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e73eb607-61b2-4576-a956-0dd6bfdf1bc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7445b3f8940244c2abe1e3f1573bcd8a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:04.361827 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e73eb607-61b2-4576-a956-0dd6bfdf1bc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7445b3f8940244c2abe1e3f1573bcd8a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:04.379960 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1454/2904] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 11:00:04 2022] DELETE /identity/v3/users/7445b3f8940244c2abe1e3f1573bcd8a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 11:00:04.570655 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce48687-9cdb-48c0-8118-93b8c5d8d5b0 service placement] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 11:00:04.584611 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce48687-9cdb-48c0-8118-93b8c5d8d5b0 service placement] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 11:00:04.648554 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0c773-7d10-43e3-a789-ab03320496b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 11:00:04.649310 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0c773-7d10-43e3-a789-ab03320496b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 11:00:04.649699 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0c773-7d10-43e3-a789-ab03320496b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 11:00:04.649978 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0c773-7d10-43e3-a789-ab03320496b1 admin admin] PATH_INFO: `/v3/projects/ad63d5a5bdb44dd6af0c25d524f38508` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 11:00:04.651207 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0c773-7d10-43e3-a789-ab03320496b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad63d5a5bdb44dd6af0c25d524f38508)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 11:00:04.653341 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0c773-7d10-43e3-a789-ab03320496b1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 11:00:04.701991 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3af0c773-7d10-43e3-a789-ab03320496b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad63d5a5bdb44dd6af0c25d524f38508', 'request_id': 'req-3af0c773-7d10-43e3-a789-ab03320496b1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:04.710595 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3af0c773-7d10-43e3-a789-ab03320496b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad63d5a5bdb44dd6af0c25d524f38508', 'request_id': 'req-3af0c773-7d10-43e3-a789-ab03320496b1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:04.717634 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3af0c773-7d10-43e3-a789-ab03320496b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad63d5a5bdb44dd6af0c25d524f38508', 'request_id': 'req-3af0c773-7d10-43e3-a789-ab03320496b1'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:04.728331 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3af0c773-7d10-43e3-a789-ab03320496b1 admin admin] The token cache is being invalidate because project ad63d5a5bdb44dd6af0c25d524f38508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 11:00:04.728621 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3af0c773-7d10-43e3-a789-ab03320496b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:04.729281 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3af0c773-7d10-43e3-a789-ab03320496b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:04.730817 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1451/2905] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 11:00:04 2022] DELETE /identity/v3/projects/ad63d5a5bdb44dd6af0c25d524f38508 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 11:00:04.734888 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73eb607-61b2-4576-a956-0dd6bfdf1bc8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 11:00:04.737099 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e73eb607-61b2-4576-a956-0dd6bfdf1bc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 11:00:04.750307 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73eb607-61b2-4576-a956-0dd6bfdf1bc8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 11:00:04.819305 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1c4a8-e6d3-416c-b3cb-e3315ad3ab11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 11:00:04.820141 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1c4a8-e6d3-416c-b3cb-e3315ad3ab11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 11:00:04.820422 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1c4a8-e6d3-416c-b3cb-e3315ad3ab11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 11:00:04.820698 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1c4a8-e6d3-416c-b3cb-e3315ad3ab11 admin admin] PATH_INFO: `/v3/users/97a484f269514769a901b18e79fd54e7` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 11:00:04.821900 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1c4a8-e6d3-416c-b3cb-e3315ad3ab11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97a484f269514769a901b18e79fd54e7)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 11:00:04.824076 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1c4a8-e6d3-416c-b3cb-e3315ad3ab11 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 11:00:04.872587 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f8c1c4a8-e6d3-416c-b3cb-e3315ad3ab11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97a484f269514769a901b18e79fd54e7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:04.881206 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f8c1c4a8-e6d3-416c-b3cb-e3315ad3ab11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97a484f269514769a901b18e79fd54e7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:04.885120 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 11:00:04.890081 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f8c1c4a8-e6d3-416c-b3cb-e3315ad3ab11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97a484f269514769a901b18e79fd54e7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:04.891682 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f8c1c4a8-e6d3-416c-b3cb-e3315ad3ab11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97a484f269514769a901b18e79fd54e7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:04.893492 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f8c1c4a8-e6d3-416c-b3cb-e3315ad3ab11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97a484f269514769a901b18e79fd54e7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:04.914894 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1455/2906] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 3 11:00:04 2022] DELETE /identity/v3/users/97a484f269514769a901b18e79fd54e7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 11:00:05.110178 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0c773-7d10-43e3-a789-ab03320496b1 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 11:00:05.112531 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3af0c773-7d10-43e3-a789-ab03320496b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 11:00:05.134296 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0c773-7d10-43e3-a789-ab03320496b1 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 11:00:05.143338 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 11:00:05.197638 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac88811-3254-4835-8af1-f4b5f3bfb5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb1a77890b95445982cf98d4ad77cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a90579cdd4284a47979efcd0ca29e39a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 11:00:05.198437 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac88811-3254-4835-8af1-f4b5f3bfb5b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 11:00:05.198576 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac88811-3254-4835-8af1-f4b5f3bfb5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 11:00:05.198717 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac88811-3254-4835-8af1-f4b5f3bfb5b7 admin admin] PATH_INFO: `/v3/projects/93903bcbf1334650bdc816450f2ecc01` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 11:00:05.199889 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac88811-3254-4835-8af1-f4b5f3bfb5b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93903bcbf1334650bdc816450f2ecc01)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 11:00:05.201897 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac88811-3254-4835-8af1-f4b5f3bfb5b7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 11:00:05.249260 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5ac88811-3254-4835-8af1-f4b5f3bfb5b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93903bcbf1334650bdc816450f2ecc01', 'request_id': 'req-5ac88811-3254-4835-8af1-f4b5f3bfb5b7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:05.256724 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5ac88811-3254-4835-8af1-f4b5f3bfb5b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93903bcbf1334650bdc816450f2ecc01', 'request_id': 'req-5ac88811-3254-4835-8af1-f4b5f3bfb5b7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:05.263268 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5ac88811-3254-4835-8af1-f4b5f3bfb5b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93903bcbf1334650bdc816450f2ecc01', 'request_id': 'req-5ac88811-3254-4835-8af1-f4b5f3bfb5b7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:05.273859 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5ac88811-3254-4835-8af1-f4b5f3bfb5b7 admin admin] The token cache is being invalidate because project 93903bcbf1334650bdc816450f2ecc01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 11:00:05.274152 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5ac88811-3254-4835-8af1-f4b5f3bfb5b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:05.274829 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5ac88811-3254-4835-8af1-f4b5f3bfb5b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 11:00:05.276396 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1452/2907] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 3 11:00:05 2022] DELETE /identity/v3/projects/93903bcbf1334650bdc816450f2ecc01 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 11:00:14.910957 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 11:00:15.270847 dv-cinder-850836-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 11:02:00.405123 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1c4a8-e6d3-416c-b3cb-e3315ad3ab11 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 11:02:00.406664 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f8c1c4a8-e6d3-416c-b3cb-e3315ad3ab11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 11:02:00.558036 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1c4a8-e6d3-416c-b3cb-e3315ad3ab11 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 11:02:00.615201 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d865b5-6e84-4e42-911b-4bfbe6fc1e9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2c5536eddd346d692822a2f3befb48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4272b859979d49c8b8f0c17ec9eb81bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 11:02:00.615820 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d865b5-6e84-4e42-911b-4bfbe6fc1e9e service placement] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 11:02:00.616118 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d865b5-6e84-4e42-911b-4bfbe6fc1e9e service placement] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 11:02:00.616371 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d865b5-6e84-4e42-911b-4bfbe6fc1e9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 11:02:00.618164 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-61d865b5-6e84-4e42-911b-4bfbe6fc1e9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 11:02:00.629324 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d865b5-6e84-4e42-911b-4bfbe6fc1e9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 11:02:00.630949 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d865b5-6e84-4e42-911b-4bfbe6fc1e9e service placement] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 11:02:00.640825 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1456/2908] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 3 11:02:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 11:02:10.636468 dv-cinder-850836-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}